CS Rarity List [WIP] - status update on page 75

[Solloby]

Thanks Law, you're a legend!

Here are the overall litter statistics for July:

Pet Count Summary
Litters
Moon Dog 1693 16%
Sun Dog 1251 12%
White July 1176 11%
Coon Dog 1164 11%
Speckle Rat 447 4%
July Rat 719 7%
Ice Rat 536 5%
Pawprint Rat 200 2%
Galaxy 2302 22%
Sorbet 1036 10%
Total 10524 100%


The Chi Test results for the litters:
Chi-Tests
Sorbets 0.449024575 0.05 Equal Rarity
Galaxies 5.75002E-28 0.05 Different Rarity
Moon Dogs 6.58941E-74 0.05 Different Rarity
Sun Dogs 1.75341E-74 0.05 Different Rarity
White July 7.40578E-52 0.05 Different Rarity
Coon Dogs 6.92857E-34 0.05 Different Rarity
Pawprint Rats 0.322198807 0.05 Equal Rarity
Speckle Rats 0.042952905 0.05 Different Rarity

The most interesting result is that the Speckle Rats have different rarities. From what I can see, Light Speckle Rat appears to be a bit more common? Odd. Once we get the Dec 18 rereleases for 08 and 09 counted, we'll be able to post the full results for July & August, hopefully with some across-litter data too!

So the project's to do list is now:
- Identify September 08 ID range
- Set up Sept 08 template
- Count Sept 08 pets
- Repeat above 3 steps for Rerelease 08 & 09 (+10 + 11?)
- Send compiled data and calculated values to volunteers for checking before final results release
- Present all of our wonderful data for July, Aug & Sept 08 rarities!

[nickjr]

-drools all over Law's script-

... So would it be realistic to run that on every pet ever from 2008 if you hard-code all the pet outcomes in? :'D (roughly 920,000 pets; the breakdown for the 2008 months we haven't done is rooouuughly:
September 2008 - a little over 100k pets
October 2008 into the first week of November 2008 - 175k-ish
November 2008 sans the first week - 175k-ish
December 2008 - roughly 300k [edit: uh. idk about rereleases])

(if it's not realistic, don't worry LOL not trying to put pressure on you at all but I'm so curious -sparkly eyes-)

[Law]

Thanks Law, you're a legend!

Here are the overall litter statistics for July:

Pet Count Summary
Litters
Moon Dog 1693 16%
Sun Dog 1251 12%
White July 1176 11%
Coon Dog 1164 11%
Speckle Rat 447 4%
July Rat 719 7%
Ice Rat 536 5%
Pawprint Rat 200 2%
Galaxy 2302 22%
Sorbet 1036 10%
Total 10524 100%

I guess I'm not surprised that the galaxy litter was more popular back then. The CEG and BEG are really pretty :o

I'm interested in seeing the exact counts for each pet. I wikipedia'd the chi test, but I still don't understand it xD;

-drools all over Law's script-

... So would it be realistic to run that on every pet ever from 2008 if you hard-code all the pet outcomes in? :'D (roughly 920,000 pets; the breakdown for the 2008 months we haven't done is rooouuughly:
September 2008 - a little over 100k pets
October 2008 into the first week of November 2008 - 175k-ish
November 2008 sans the first week - 175k-ish
December 2008 - roughly 300k [edit: uh. idk about rereleases])

(if it's not realistic, don't worry LOL not trying to put pressure on you at all but I'm so curious -sparkly eyes-)

Haha yes that's right! The main thing I'm worried about is the pressure it'll put on chickensmoothie's servers. Maybe I could do it in chunks at a somewhat reasonable pace?

[nickjr]

LAST EDITED 40 MINUTES AFTER TIMESTAMP

I guess I'm not surprised that the galaxy litter was more popular back then. The CEG and BEG are really pretty

I'm interested in seeing the exact counts for each pet. I wikipedia'd the chi test, but I still don't understand it xD;

Makes me wonder if Tess keeps the litter %s of the original 2008 adoptions or if she sets them to be closer to what they would be if they were released with adoption limits xD
(... bleh I worded that badly--so e.g. there are only two coon litter outcomes (coon and noncoon), and there are also only two galaxy litter outcomes (BEG and CEG), but from the counts Sol posted of the litters, there are literally 2x more galaxies total than coons/noncoons total from the July [edit: and August?] 2008 adoptions; today, adoption limits generally correlate to the number of outcomes, so if the galaxy and coon litters had limits, then their limits should be roughly the same... so I wonder if Tess sets the %s such that the chances of getting either galaxy is 2x as much or only the same as getting either the coon or noncoon

I haven't been paying attention to rereleases, so I don't even have a general idea lol)

Chi-square test, quick and dirty: I'm pretty sure it's a test to show the chances of getting your actual distribution or something more extreme, given your expected/hypothetical distribution (e.g. if you are expecting a 50-50 distribution but actually have a 25-75 distribution, you want to do a chi-square test to see the chances of getting 25-75 or 24-76 or 23-77 or ... or 1-99 or 0-100 combined [not just 25-75] are low; if the chances are low, then your hypothesis of 50-50 is wrong and the actual distribution is something else

you can see how it's a pain to calculate by hand when you have a bunch of categories lolol)

Haha yes that's right! The main thing I'm worried about is the pressure it'll put on chickensmoothie's servers. Maybe I could do it in chunks at a somewhat reasonable pace?

-MAJOR SPARKLY EYES-
You initially posted in here with almost 5k pets done. Did you do that almost all at once or did you do that a little at a time over a few days or? Because we could do that just 20 times to cover the 100k-ish pets in September

edit: Then again... how many pets have we counted (read: loaded the images for) in the last few days or since we started on July 2008? Could use that as a reference and then just space out the times of day when the script runs

edit2: 2 weeks for 12k pets? Actually maybe it would be faster to use how long it took Law to get the almost 5k pets done as a reference lol

(that being said, 2 weeks for 12k pets is still pretty darn awesome; it's just that... there are 100k-ish pets from September 2008 lol gonna have to go choose pet ID ranges again if we can't do 100k too fast)

edit3: Other pets to keep in mind besides the regular monthlies?:
> Tess bday pets; count or no? they're not main list but they're Septembers on the List unless I'm seeing things, and since we're counting Septembers anyway (that's what I'm assuming from the to-do list on the previous page)...
> warriors and tess horses? not to count but more to keep in mind (remember our ID ranges in August 2008 filled with 2nd gens and bwolves lol); will check when the batches came out old news posts: Tess horses out on Sept 21 and Oct 6, Blackstar (single warrior) out on Sept 21 (other warriors released in July/August or in 2009)
> the last two front page bwolves? the white/blue and white/green ones (also just to keep in mind); will check this too bwolves out on Sept 21
> vampire x mismatch 2gen litter came out in October, I think November; that's the last 2008 2nd gen litter

edit4: um, ignore everything I said that mentions October 2008/November 2008 because I think we're not counting those lol
(... unless we want to
but nah let's go to the rereleases after we do September 2008)

[Law]

Makes me wonder if Tess keeps the litter %s of the original 2008 adoptions or if she sets them to be closer to what they would be if they were released with adoption limits xD
(... bleh I worded that badly--so e.g. there are only two coon litter outcomes (coon and noncoon), and there are also only two galaxy litter outcomes (BEG and CEG), but from the counts Sol posted of the litters, there are literally 2x more galaxies total than coons/noncoons total from the July [edit: and August?] 2008 adoptions; today, adoption limits generally correlate to the number of outcomes, so if the galaxy and coon litters had limits, then their limits should be roughly the same... so I wonder if Tess sets the %s such that the chances of getting either galaxy is 2x as much or only the same as getting either the coon or noncoon

I wasn't here during 2008, but I seem to recall reading a post about how there weren't limits back then? Or was that a different case/wasn't the case at all? Users had to get their pets manually added to their accounts or something.

I haven't been paying attention to rereleases, so I don't even have a general idea lol)

same haha

Chi-square test, quick and dirty: I'm pretty sure it's a test to show the chances of getting your actual distribution or something more extreme, given your expected/hypothetical distribution (e.g. if you are expecting a 50-50 distribution but actually have a 25-75 distribution, you want to do a chi-square test to see the chances of getting 25-75 or 24-76 or 23-77 or ... or 1-99 or 0-100 combined [not just 25-75] are low; if the chances are low, then your hypothesis of 50-50 is wrong and the actual distribution is something else

you can see how it's a pain to calculate by hand when you have a bunch of categories lolol)

your explanation makes sense (thanks!) :o
Do you happen to know what the numbers mean in "Sorbets 0.449024575 0.05 Equal Rarity
Galaxies 5.75002E-28 0.05 Different Rarity"?

Haha yes that's right! The main thing I'm worried about is the pressure it'll put on chickensmoothie's servers. Maybe I could do it in chunks at a somewhat reasonable pace?

-MAJOR SPARKLY EYES-
You initially posted in here with almost 5k pets done. Did you do that almost all at once or did you do that a little at a time over a few days or? Because we could do that just 20 times to cover the 100k-ish pets in September

For reference, it took 10 minutes to get the 5k pets done, including getting the results into the spreadsheet. So I guess 100k pets wouldn't be too bad. Might be wise to space that out of course.

edit4: um, ignore everything I said that mentions October 2008/November 2008 because I think we're not counting those lol
(... unless we want to
but nah let's go to the rereleases after we do September 2008)

I'm down for some rereleases after Sept 08. I've always wanted to know what the chances were for getting a list pet.

[nickjr]

LAST EDITED 10 15 MINUTES AFTER TIMESTAMP

(too lazy to quote you and then break up the quote LOL)

Yeah, no adoption limits until around November 10, and pets couldn't be added onto accounts until late September (when I joined! haha see this girl's adoption date xD). But there wasn't really a huge increase in adoptions until October; until then, most people just adopted a few pets per litter to see what they would get, and they mostly just adopted the ones they wanted; the idea of having a complete collection wasn't quite as intense back then

I haven't used the chi-square test in Excel, but I'm guessing this is the format of the data that Sol is posting:
[Litter] [% in decimal form of getting a distribution result as extreme as what we have, given the expectation that all outcomes are equal] [% threshold in decimal when we want to say "this extremity is too rare to happen by chance"] [final outcome: outcomes are of equal rarity or no]

e.g. "Sorbets 0.449024575 0.05 Equal Rarity"
That 0.449... is the chance of getting a distribution that is at least as extreme as the distribution we have in our data (Sol hasn't posted this, probably to prevent people from jumping on the numbers too early) if we assume that each of the 4 outcomes is equally likely
That 0.05 is the threshold where we say "okay this is too unlikely" (common in statistics to have either 5% or 1% as this threshold; might seem kinda low to people, but this way, if we say something is statistically significant, then it's probably significant LOL*)
So given the assumption/hypothesis that all sorbet outcomes are of equal rarity, we have a 45% chance of getting our actual data, which is..... pretty darn high lol waaaay higher than the 5% threshold
(E = exponent of 10, like on scientific calculators--e.g. E-28 means you multiply the number before that with 10^-28)

(We can actually assume/hypothesize that the outcomes are not equal, but we would need numbers for that lol the easiest is just to assume that the outcomes are equal, especially since we have no idea what unequal %s we should assume, but in stats class you might do an exercise with candies or fruit snacks where the manufacturer has actually published stuff like "only 3% of your M&Ms will be blue" or something, and so you would assume those unequal %s instead--but I digress

* digressing some more: decreasing the threshold means decreasing one type of error, but on the flip side, increasing the threshold means decreasing the likelihood of a different type of error...here for more info)

Oh wow o: Could have the 100k pets done in like a week or only a few days... run it four times a day for 5k pets each time and we'd be done in 5 days (doesn't always haveto be you, of course) for... basically a census of the non-rerelease Sept 08 pets
That's mildly terrifying

(anyone please correct me if I got sth wrong; this is what I remember from AP Stat and having my teacher verify my understanding)

[Solloby]

Wups, sorry for dumping the stats without explaining! I didn't post the actual numbers because the community agreed after we finished the August count that we should wait for the Dec 18 rerelease data first to add them to our results.

To explain the Chi test, this is what I was attempting to do.
We know that the Sorbets are equal in rarity, because Tess told us so back in 2008. Here is our July data for the Sorbets:
Pink Sorbet 241
Blue Sorbet 258
Yellow Sorbet 259
Green Sorbet 278

If you add up the 4 amounts then divide by 4 to get the average, you get 259.
Our null hypothesis is that sorbets are statistically equivalent in rarity, which means we should have 259 of each with a 5% room for error due to randomness. The purpose of the statistical test is to determine whether this is true.

This Chi test results I posted included:
Sorbets 0.449024575 0.05 Equal Rarity

0.449... was the result of the Chi test (Excel statistical function) that I ran using the four above values and comparing them to 259. The question the Chi test was asking was, is the actual number of counted Sorbets the same as the expected number of 259? The 0.05 was the room for error. "Equal Rarity" is produced by a simple Excel formula if the Chi test result is greater than 0.05 ("Different Rarity" is the output if the number is less than 0.05).

Hopefully I explained ok, and interpreted the statistical test correctly?

-------------

For the September count, what is a good ID range? When do we start and end? I think September will be easier than July/August because we didn't have pets being released in the middle of the month I don't think.

[Law]

Oooh, I see now! It's interesting how that the speckle rats got a low probability, but not that low compared to the other different rarity litters. Thanks for the explanation nickjr and Solloby :D

Based on the archive, it looks like the cake dogs were released on 9-10 (at around #145000). Perhaps 120k-170k? I'm up for doing all 100k if needed.

[Fizzlebee]

@Law what would we need to do to prepare your system for the September count? Find an example of each pet we're interested in counting?

[Law]

@Law what would we need to do to prepare your system for the September count? Find an example of each pet we're interested in counting?

Yep, just need a picture of each outcome (doesn't matter if it includes the owner's name at the bottom). I would also like the names as well :)