13

According to the About page:

Public betas typically last from 60 to 90 days, although they can go on longer.

Our site appears healthy yet it has now been in public beta for 117 days.

  • Is there anything we can or should be doing to move the process along?
  • Is moving the process along at this point desirable?

3 Answers 3

7

We... Probably should change that text.

The original design called for a site to either graduate or be closed within 90 days. It quickly became clear that a lot of decent sites can take a fair bit longer to either reach sustainable mass or clearly fail:

How long can a site stay in beta?

The simple answer is, it takes as long as it takes. We’ll wait. If a site needs more activity, go out and evangelize it. As long as your site shows steady progress and continues to make the Internet a better place to get expert answers to your questions, it will march on. We don’t want to kill a site because it hasn’t reached full status in 90 days. Nor do we want to set a hard 90-day limit and launch a site too soon.

There’s more to the health of a Stack Exchange site than having a lot of questions and answers. There’s an economy to the site with reputation as its currency, and voting drives that economy. A site absolutely needs on-going, sustained voting to build a class of leaders that help run and govern the site. Without leadership, there can be no community.

So from this point forward, the graduation date of a site will depend heavily on having enough users with sufficient reputation to properly lead and govern the site. It’s much more important to graduate a site when it has become self-sustaining, and has established a healthy community of avid users, closers, and editors — rather than imposing an arbitrary 90-day limit.

Thus, the order of launch will favor those beta sites which have achieved the most “excellent” ratings on our Area 51 stats panel. For everyone else — keep going!

See: Mathematica Area 51 stats

FWIW: we generally evaluate each site for graduation-readiness every 60 days. We're currently about two weeks behind schedule, so I can't say if Mathematica is ready yet, but the stats look pretty good - the only thing slightly worrying is the number of users with high reputation (thresholds for various privileges change upon graduation, necessitating a reasonable number of users with at least 500, 2K, 3K, and 10K reputations in order for the site to operate properly). This also ties into the ability to hold moderator elections, which are generally held shortly after graduation.

9
  • 2
    "only thing slightly worrying is the number of users with high reputation"... have you actually looked at the list of users by reputation (see rep table here)? Most graduated sites don't have that many (and I'm not talking about the "oops, we graduated webapps.se" kind of sites). I'm not asking for hard numbers, but some kind of feedback and engagement from the SE team would be appreciated rather than stone silence
    – rm -rf Mod
    May 22, 2012 at 1:03
  • 2
    Yes, I did; 'twas a bad choice of words on my part. I guess I should clarify: Mathematica.SE looks great - the only question is whether it's ready to graduate now or soon. The number of high-rep users is "worrying" only in the sense of "would we hamper your growth by graduating you now" - some of the folks who can and are editing or closing right now wouldn't be able to immediately after graduation, for instance.
    – Shog9
    May 22, 2012 at 1:08
  • Regarding feedback: watch for a quality evaluation in the near future.
    – Shog9
    May 22, 2012 at 1:11
  • Thank you for the feedback. As a 5k+ user on SO, I am personally content with that level of interaction, even though access to the mod-tools has been interesting. However, I think we compare relatively favorably to RPG.se, especially at the 10k+ level (4 non-mod v. 6 at RPG). While at the 3k level, we only have 29 compared to their 40, we have 7 users who are reasonably close to that mark. Plus there are a number of newer users who are moving up the ranks quickly. At this point, I think we will do just fine if we graduate.
    – rcollyer
    May 22, 2012 at 1:41
  • 1
    I would add, not only do we have 29 users who would still have close privileges, we have at least 22 on my count, who actually have voted to close a question that was eventually closed.
    – Verbeia
    May 22, 2012 at 2:15
  • 2
    ~40 who've voted to close all told, @Verbeia (not all votes have led to closures, and some closures are still pending. BTW: seriously impressed with the data you've kept on this site's progress!
    – Shog9
    May 22, 2012 at 2:19
  • @Shog9 Also more stats here. The /review page shows a different view for different folks. For instance, I don't see any posts in there (probably because I've cast a vote on most of them or reviewed and dismissed it)
    – rm -rf Mod
    May 22, 2012 at 2:22
  • 1
    @Shog9 thanks for your kind words. (I'm an economist - I can't help wanting to graph things.) I only tallied up the votes on actually closed questions, of the closed and deleted questions I can see.
    – Verbeia
    May 22, 2012 at 2:31
  • 1
    One additional comparison: Skeptics. Of the 10k+ users, only 2 are not moderators. Also, they only have 7 more 2k+ users than us, and we actually are beating them on 3k+ users, by 2. (: However, they have quite a few users that are nearly at 2k, we do not have the same depth, as of yet. (And, yes there are quite a few of us who like numbers. :)
    – rcollyer
    May 22, 2012 at 2:47
14

What I generally find puzzling is why so many of us want to graduate a.s.a.p. What is it that all these folks anticipate so impatiently that would happen upon graduation? I mean, does everyone who is in favor of graduating now realize what it'll be like? Ok, let me tell you what's my picture:

  • Many more basic, lower quality questions

  • Many more beginner users

  • Much less consistent voting (not only different questions / answers to them would receive relatively very different weights - which experts often would find inadequate, but also within the same question, truly great answers won't always win)

  • Much heavier load on all experts and advanced users to keep things under control, and most importantly, preserve the coherence of our community, the culture we've developed, etc.

To my mind, at any given time, we (as a community having experts, advanced users, intermediate users and beginners) can handle only so many basic questions, and beginner users who ask them. As we grow, we get more of the intermediate-level users who share our culture and are ready to take some of that load off the core of experts. We have now a great proportion / ratio of experts / advanced users / intermediate users / beginners, and I think this is one big reason why we are doing so well as a community. In other words, It is this power law with a much steeper power which makes us stand out (because it reflects an unusual number / proportion of experts willing to help). However, the other side of this is that, if you continue the curve to the other end, we have less room for beginner users than other communities (I mean the proportion, not the absolute number), at any given time (this room grows as we grow). This is a speculation on my side, but IMO power law is important, since it happens naturally in scale-free networks, which are characteristic descriptions for systems growing organically. Graduate now, and you will create a second exponent, with, I think, disastrous consequences for us.

Mathematica SE, as well as other similar SE sites, is a modern system of knowledge transfer (thanks to the SE team). We are attracted and willing to spend our time here because we see that this system is effective. It is effective because we have a coherent community with a certain culture, ways of doing things, informal rules, etc. But we are still very young as a community. Scaling this effectiveness to larger community size is a non-trivial task. For knowledge transfer to be effective at the larger scale, we need more intermediate users willing to participate and serving as a bridge between experts and more beginner users. It is not accidental that, while we are doing more than well in numbers of experts / advanced users, our number of intermediate users is just a fraction more than is requested. Besides that, we need also more questions already answered, more "canonical" / "generic" answers (which serve to prevent duplication of efforts and fragmentation of knowledge), more people who are aware of past questions and generally carry our common knowledge, etc.

To summarize, I have a very strong feeling that we are not yet ready for rapid expansion. All those parameters which SE team introduced to measure the degree of our readiness for graduation are there for a reason. Right now, I think that the longer we stay in Beta and accumulate critical mass (in many aspects), the better it would be for us (of course, I don't mean staying in Beta for years). We have previously happily existed on Stack Overflow for years, and, while there were certain problems and inconveniences due to restrictions of SO format and misunderstandings with users coming from other tags, I think we were doing great. Right now, we already accumulate new users at the rate unthinkable for the SO Mathematica tag, and have other signs of rapid but healthy growth.

So, what's the rush now? My two cents.

EDIT

I highly recommend to read this blog post, coming from an SO user who was consistently within top 10-20 SO users for extended periods of time while still on SO.

29
  • 1
    Please note that I did ask: "Is moving the process along at this point desirable?" I too have wondered at the impatience for graduation. Nevertheless I would like to know your reasoning for the bulleted points upon graduation. This is an open beta after all, right?
    – Mr.Wizard
    May 22, 2012 at 11:48
  • @Mr.Wizard Well, I took this opportunity to summarize what I noticed as reactions of many people. Sorry if I misused your question, I did not mean you in particular - but I think this answer is still rather on topic. I'd rather make it a comment actually, but it is too long for that. As to my reasoning - yes, we are open, but right now I'd just do anything possible to keep it the way it is, and not make us more open than we already are, for the time being. May 22, 2012 at 11:56
  • Leonid, that vote is mine. I'm not really understanding how graduation would make us more open -- please explain. Also, why will being more open cause an increase in, how do I say, less-than-motivated users?
    – Mr.Wizard
    May 22, 2012 at 11:59
  • 2
    @Mr.Wizard "that vote is mine" - I figured that much :). Thanks. The graduation may make us more open since I presume that graduated sites generally get more attention, and also lots of people tend to consider a status of the site, and the "graduated" badge may mean a lot to them. Being more open will likely increase the inflow of the mentioned users, just because they will be more likely to "discover" our site. I am actually puzzled that this did not happen already at a much larger scale, and the only explanation I have is that we still have high entry barrier. Graduation will lower it. May 22, 2012 at 12:06
  • 1
    My theory is that there aren't as many "riffraff" users of Mathematica as you fear. I hope I'm right, but I wouldn't bet against you.
    – Mr.Wizard
    May 22, 2012 at 12:21
  • 1
    @Mr.Wizard I am afraid that you are wrong here ( I wish you are right :)). The problem is that, while programming languages have an inherently non-zero (and, for some languages, rather high) entry barrier, Mathematica is much more than a language. While on SO, we were safe, because non-programming questions were off-topic, while as a language Mathematica has a rather high entry barrier (which is a mixed blessing for it IMO). But now, we are open to all kinds of basic stuff, and let's keep in mind that Mathematica is also an environment used by students at universities and high schools. May 22, 2012 at 12:28
  • I concede the argument.
    – Mr.Wizard
    May 22, 2012 at 12:30
  • I'm not sure I can trust anything the author of the blog you link says: he is using a drop shadow on body text =headache=
    – Mr.Wizard
    May 22, 2012 at 12:36
  • 2
    Leonid, from the start of the beta, I've been telling people not to worry about graduation after exactly 90 days or so. I think I share some of your concerns (but not all). My main beef is with the SE team, who barely (almost never) communicate with their own sites. They culled 4 sites last month, and the first time that the users of those sites and the mods there heard about it was from the blog post — how uncourteous is that?! So yes, I am impatient, but more so for some communication rather than graduating right now.
    – rm -rf Mod
    May 22, 2012 at 13:17
  • @R.M. I wholeheartedly agree. I was quite shocked by the closure of those sites myself, and not by the mere fact of it but by the way it was carried out. If you check my reply to the blog post I linked to, you will see that I have similar concerns about SE. But, SE is the way it is, and currently there is not much we can do about it. But I think, we can and should set up the model for Mathematica SE which we think is right. I already mentioned once that the best way is when it is enforced by the system and made a part of the "language", but our own wise decisions can also go a long way. May 22, 2012 at 13:25
  • Thanks Leonid, that's very kind :) I've learnt a lot from just passively following people and seeing others' solutions/approaches.
    – rm -rf Mod
    May 22, 2012 at 14:30
  • @Mr.Wizard "I too have wondered at the impatience for graduation." For me, I think it makes a little bit harder for them to shut us down, as I still feel a little burned by the shutdown of the first Area51 proposal. However, my current concern is our tail. At the top end, we are as solid as two of the more recent graduates, but we don't have the intermediate users that we would like. They will ultimately drive the site, not the ones at the top. How do we get more of them? time, unfortunately.
    – rcollyer
    May 22, 2012 at 14:39
  • 2
    @Leonid You make some good points about why it would be best not to have a sudden influx of new users. But why do you think graduation would trigger that? As far as I can tell, graduation is just a promise that they'll try not to shut down the site, and brings along an increase in rep thresholds and a custom site design.
    – Szabolcs
    May 22, 2012 at 15:02
  • 1
    @OleksandrR. Totally agree with all you said. One thing though: you repeatedly classified yourself as belonging to intermediate users, while in my view, you are one of the top experts on this site. I learned quite a few things from you, and I am sure I am not alone. May 25, 2012 at 20:29
  • 2
    @LeonidShifrin: thanks for your kind words; it's nice of you to say so. But let me clarify: though I'm reasonably capable in some areas by now, I still find myself learning new things at too high a rate to feel comfortable thinking of myself as any sort of expert. :) I know I have a long way to go in some areas--front end programming in particular. I had no idea at all that you can have context-dependent or programmatically determined syntax coloring until recently! That's the beauty of this community, IMO--Mathematica is huge, but we all have different specialisms and experience to share! May 30, 2012 at 0:59
6

As I pointed out above, our core of experts is almost as numerous as those found on RPG, Skeptics, and even User Experience, and it is growing. The difficulty is we do not have the same depth of field as those three sites. They have existed longer than us, so that is not surprising. But, I believe we need more intermediate users to make graduation viable, and there is only one way for that to occur: time. Time, and of course our selfless (shameless?) self-promotion on sites like reddit.com.

That said, I would like to propose a graduation date that will give us time: October 17. That way we can announce our graduation at the Wolfram Technology Conference, and possibly send some of our users to it. Thoughts?

8
  • Why do you think that we need more publicity now? Intermediate users will find us in any case, and I am very much in favor of promoting us during Wolfram Technology Conference (because it is a highly targeted audience), but why do you think we should actively promote ourselves elsewhere? What kind of audience do you plan to attract? We won't win many programmers (particularly young / creative ones) without a free and well-documented (or even open-source) implementation of core mma. We can win people from academia, but I am not sure we'll get the right ones (profs / postdocs / smart students). May 22, 2012 at 15:50
  • As to the graduation date, I agree that the coming Fall (perhaps October indeed) should be the good time for it. May 22, 2012 at 15:52
  • @LeonidShifrin I'm looking at the visits/day which is one of the measures that is looked at. In addition to providing temporary spikes, promotion has increased our avg visits/day, too. Are these the users we want? I don't know. But, it is a way of getting people in the door which may not have happened otherwise. The beta sites are somewhat hidden relative to the graduated ones. As I said, however, time is the only way we'll get there.
    – rcollyer
    May 22, 2012 at 16:02
  • "As I said, however, time is the only way we'll get there" - that's what I also think. Time, plus all the measures WRI will be able to take, to attract more of the creative and smart people to Mathematica. May 22, 2012 at 16:08
  • @LeonidShifrin but, I don't think self-promotion is a bad thing. Some of our questions and answers are stellar, and have brought more people in. And, if it is only in views, they may be tempted to stick around. We, as the expert users, will have to work on keeping our culture together as new users are indoctrinated, but that is why I think we need more intermediate users (in rep, if not expertise). They can run some of the interference for us.
    – rcollyer
    May 22, 2012 at 16:11
  • Perhaps you are right. Stellar questions / answers are more likely to attract smart and enthusiastic people, which is what we need. May 22, 2012 at 16:27
  • @LeonidShifrin of course I could be wrong about reddit being a good forum to advertise. :P
    – rcollyer
    May 22, 2012 at 17:31
  • 1
    I just counted and there are 18 users with rep between 1000 and 2000. 15 of them have been seen on the site in the past two days, 14 within the past day.
    – Verbeia
    May 23, 2012 at 4:46

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .