Some v94 stats

Here we report bugs in the beta.
User avatar
Petter
Site Admin
Posts: 662
Joined: Mon Dec 08, 2014 11:03 pm

Some v94 stats

Post by Petter » Sun Feb 18, 2018 1:50 pm

Good news everyone!
I've collected some stats from the usage of v94, all available at http://wiki.cell-lab.net/Cell_Lab_user_statistics

As can be seen in the graph below it seems like the challenge completion rate has gone up significantly with the rather small changes made between v74 to v94. Particularly the more difficult challenges show almost a doubling of the completion rate. I'm not sure what the reason might be, probably a combination of
  • the tip to check out video tutorials that pop up after a while
  • the new fun and easy challenges in the beginning that motivate users to continue
  • some improved phrasings
We'll need some more time to make 100% sure this is not some effect of the update. I've waited a week after the 6% rollout until data collection started.
Image
User avatar
Alast
Posts: 2216
Joined: Thu Jul 02, 2015 9:15 am

Re: Some v94 stats

Post by Alast » Sun Feb 18, 2018 7:42 pm

That is great news! :-) Looking forward to more data.
Perfection hasnt reached me yet, but its trying hard!
User avatar
Petter
Site Admin
Posts: 662
Joined: Mon Dec 08, 2014 11:03 pm

Re: Some v94 stats

Post by Petter » Sat Mar 03, 2018 10:41 pm

I've now updated the wiki with a couple weeks more data giving more accurate statistics. I've also added these three lines to the median win time plot:
Image
They have been positioned to intersect at challenge number -20 and give 25% of v94 median win times in each sector. I'm thinking they could be used as an objective way of deciding challenge difficulties? The ones with yellow bars below the lowest line: undergrad, the one between the lowest and the middle line: phd, etc. I've put some slope to the lines since I think the later challenges should naturally take a bit longer, the user should have more patience by then. Quite literally a learning curve, and we can measure it's steepness :P. This would result in the difficulty labels below. I'm inclined to changing the difficulties to this, maybe with some changes to the parameters. What do you think about it?
  • Algae: undergrad
    Algae II: phd
    Algae III: undergrad
    Macrophages: undergrad
    Macrophages II: undergrad
    Colors: mad scientist
    Breeding: phd
    Coexistence: superhuman AI
    Macrophages III: undergrad
    Macrophages IV: phd
    Sunlight: mad scientist
    Scavenger: phd
    Seed: undergrad
    Sunlight II: phd
    Macrophages V: phd
    Decimation: undergrad
    Breeding II: superhuman AI
    Infestation: mad scientist
    Countermeasure: mad scientist
    Conquering: mad scientist
    Floaters: phd
    Floaters II: phd
    Parasite: phd
    Sterilization: mad scientist
    Mosh pit: mad scientist
    Slippery Slope: undergrad
    Sterilization II: undergrad
    Moss IVF: mad scientist
    Floaters III: mad scientist
    Harsh Conditions: undergrad
    Harsh Conditions II: undergrad
    Symbiosis: phd
    Slippery Slope II: phd
    Infection: undergrad
    Maze: phd
    Infection II: superhuman AI
    Noncommutative viruses: mad scientist
    Infection III: phd
    Gene Therapy: superhuman AI
    Digestion: superhuman AI
    Digestion II: mad scientist
    Population Bottleneck: phd
    Adaptation: phd
    Paleogenetics: undergrad
    Tip the Balance: superhuman AI
    Keep the Balance: superhuman AI
    Balloons: mad scientist
    Friction: mad scientist
    Nervous: undergrad
    Friction II: undergrad
    Scarce Conditions: mad scientist
    Scarce Conditions II: undergrad
    Flat Earth: superhuman AI
    Scarce Conditions III: phd
    Digestion III: superhuman AI
    Predator: phd
    Deception: superhuman AI
    Food delivery: superhuman AI
    Move it!: mad scientist
    Extermination: mad scientist
    Discrimination: superhuman AI
    Avoidance: mad scientist
    Avoidance II: superhuman AI
    Nanobot: superhuman AI
    Desert: superhuman AI
    Desert II: superhuman AI
Noughtypixy
Posts: 234
Joined: Tue Jan 31, 2017 1:30 pm

Re: Some v94 stats

Post by Noughtypixy » Sun Mar 04, 2018 10:34 am

I would swap these difficulties around
Digestion: superhuman AI
Digestion II: mad scientist

Digestion ii definitely more difficult.
beware the pixys
User avatar
Alast
Posts: 2216
Joined: Thu Jul 02, 2015 9:15 am

Re: Some v94 stats

Post by Alast » Sun Mar 04, 2018 12:20 pm

I think some objective measurement of difficulty will definitely help labeling more acccurately! Let's go for it :)
Perfection hasnt reached me yet, but its trying hard!
User avatar
Petter
Site Admin
Posts: 662
Joined: Mon Dec 08, 2014 11:03 pm

Re: Some v94 stats

Post by Petter » Sun Mar 04, 2018 12:22 pm

Hm, yea i guess it is in absolute terms harder, but once users have finished digestion I, it seems like it is not as hard anymore. Does that make sense?
User avatar
Petter
Site Admin
Posts: 662
Joined: Mon Dec 08, 2014 11:03 pm

Re: Some v94 stats

Post by Petter » Sun Mar 04, 2018 12:38 pm

Alright! I'll probably need to update these a bit as it is rolled out to more people, I still don't have so much statistics on the harder challenges.
Btw, it seems like both tut 4 and balloons are too hard, many people don't finish them. I'm not really sure why, the tutorial should be quite straightforward. Perhaps there is some issue with e.g. the Russian translation, making up 20% of users.
fungus3
Posts: 112
Joined: Sun Oct 09, 2016 1:15 pm

Re: Some v94 stats

Post by fungus3 » Sun Mar 04, 2018 5:45 pm

Here's my thoughts of difficulty

Einstein difficulty is between mad scientist and superhuman AI difficulties.
  • Algae: undergrad
    Algae II: undergrad
    Algae III: undergrad
    Macrophages: undergrad
    Macrophages II: undergrad
    Colors: mad scientist
    Breeding: phd
    Coexistence: superhuman AI
    Macrophages III: undergrad
    Macrophages IV: phd
    Sunlight: mad scientist
    Scavenger: phd
    Seed: undergrad
    Sunlight II: phd
    Macrophages V: phd
    Decimation: undergrad
    Breeding II: superhuman AI
    Infestation: phd
    Countermeasure: mad scientist
    Conquering: mad scientist
    Floaters: phd
    Floaters II: phd
    Parasite: phd
    Sterilization: mad scientist
    Mosh pit: mad scientist
    Slippery Slope: undergrad
    Sterilization II: undergrad
    Moss IVF: mad scientist
    Floaters III: mad scientist
    Harsh Conditions: undergrad
    Harsh Conditions II: undergrad
    Symbiosis: phd
    Slippery Slope II: phd
    Infection: undergrad
    Maze: phd
    Infection II: mad scientist
    Noncommutative viruses: mad scientist
    Infection III: phd
    Gene Therapy: Einstein
    Digestion: superhuman AI
    Digestion II: mad scientist
    Population Bottleneck: phd
    Adaptation: phd
    Paleogenetics: undergrad
    Tip the Balance: Einstein
    Keep the Balance: superhuman AI
    Balloons: mad scientist
    Friction: mad scientist
    Nervous: undergrad
    Friction II: undergrad
    Scarce Conditions: mad scientist
    Scarce Conditions II: undergrad
    Flat Earth: Einstein
    Scarce Conditions III: phd
    Digestion III: superhuman AI
    Predator: phd
    Deception: mad scientist
    Food delivery: Einstein
    Move it!: mad scientist
    Extermination: mad scientist
    Discrimination: superhuman AI
    Avoidance: mad scientist
    Avoidance II: superhuman AI
    Nanobot: superhuman AI
    Desert: Einstein
    Desert II: superhuman AI
[/quote]
User avatar
Alast
Posts: 2216
Joined: Thu Jul 02, 2015 9:15 am

Re: Some v94 stats

Post by Alast » Sun Mar 04, 2018 9:35 pm

That's subjective, though ;-) We're aiming for objective measures.
Perfection hasnt reached me yet, but its trying hard!
Icehawk78
Posts: 7
Joined: Thu Mar 08, 2018 6:56 pm

Re: Some v94 stats

Post by Icehawk78 » Thu Mar 08, 2018 7:14 pm

Petter wrote:Good news everyone!
I've collected some stats from the usage of v94, all available at http://wiki.cell-lab.net/Cell_Lab_user_statistics

As can be seen in the graph below it seems like the challenge completion rate has gone up significantly with the rather small changes made between v74 to v94. Particularly the more difficult challenges show almost a doubling of the completion rate. I'm not sure what the reason might be, probably a combination of
  • the tip to check out video tutorials that pop up after a while
  • the new fun and easy challenges in the beginning that motivate users to continue
  • some improved phrasings
We'll need some more time to make 100% sure this is not some effect of the update. I've waited a week after the 6% rollout until data collection started.
One major caution to this - v94 is an opt-in beta test, so you have a massive level of self-selection happening here. That is, people who find the game significantly more engaging are more likely to find the forums and sign up for the beta, and those people are also thus more likely to be good at the game as opposed to someone who just installed it, tried it out, and then uninstalled and/or left it and didn't load the app since then.
Post Reply