CS Rarity List [WIP] - status update on page 75

My gallery · by **daryll** » Tue Jun 12, 2018 7:48 am

Summer 2017 - https://static.chickensmoothie.com/pic. ... 680B578640
Summer 2016 - https://static.chickensmoothie.com/tran ... O6Ac8_tGqg

Yes, these will trigger if found; what I could do is add another check by looking for

Code: Select all: id="petimg"

found in the same line as the image
02:57 PM:This is added, as long as the banner doesn't have the HTML tag (id="petimg") it will work during events

The banners don't show up outside of events, though. So maybe the program can just be run outside of events if you don't want to add additional checks to the program.

This is also another option

I assume that the deleted pets are identified by process of elimination (having none of those string constants on the page)?

Correct, deleted pet pages do not contain the strings the program is searching for, so the page is ignored

I'm not sure how much this suggestion will help, but maybe you can narrow down the key/legend of pet types of compare against based on a pet's adoption date*? For example, if the pet was adopted in March 2009, then you don't have to compare it against July 2008 pet types like the Sunjewel, Moonswirl, etc. right?** (Exact boundaries might be difficult to determine, especially for 2008 pets, and you'll have to be comparing to basically the full list for any December 18 pet as well as some December 19, 2009 and December 24, 2011 and December 25, 2011 pets, but still)

This can be probably be done, I can create a Date class to store the month, day, year and compare them.

My gallery · by **lil rascal** » Wed Jun 13, 2018 12:39 am

Just something I thought of if limiting to dates is that it means the store pets, that can be adopted at any time, are not going to be accurately counted.

My gallery · by **daryll** » Wed Jun 13, 2018 7:37 am

Is there a way to automate the file names or something? Like maybe have something go through the files in chronological order of when they were saved, and rename every file to be either the same (to get Windows to add the (1) (2) etc) or directly rename to what you want?

I found an external library that can grab webpages' HTML code. That way I don't have to download anything and have the program be partially automated.

The only manual input needed is determining the start ID and how many pages to look-up

Code: Select all: //Change this to determine where to start searching (1 = ID#1) static int pagenum = 55826397; //Change this to how many pages you want to look up (10 = 10 pages) static final int numberOfPages = 105;

Also: I need hard-coded reference links to identify the types of pets; I noticed that pets with the same image use the same link, but if their color is different the key in the URL is completely different. If anyone could help me with identifying that would be great

Note: This program is operating on the default skin of the site, I cannot change it as I can't actually see what the page looks like

So far the output will be a .txt file, the formatting is the following:

Code: Select all: ID: 55826398 Type: https://static.chickensmoothie.com/pic.php?k=651644407085DA3DA2FE7AD06B92A44F&bg=e0f6b2 ID: 55826399 Type: https://static.chickensmoothie.com/pic.php?k=F43A3FCB6C16BDBDF3A87B5E5E61808C&bg=e0f6b2 ... Total looked up: 105 ONLINE: 104 OFFLINE: 1 RANGE: [55826397, 55826502] IDs DELETED: 55826404

With the type being the pet type, but the image URL is a placeholder until I can actually identify pets

I can also grab the adoption date/owner name/age if needed

06/13 02:27 PM: Edited formatting to current state

06/13 06:53 PM:I did a trial run checking IDs [1,1000] - it took about 18 minutes to complete, and only used 600MB of RAM during the trial. I would assume if I checked [1,2000] that RAM usage would hit ~1300MB and take about half an hour

Code: Select all: RANGE: [1,1000] TOTAL: 1000 ON-ACCOUNT PETS: 323 OFF-ACCOUNT PETS: 640 DELETED PETS: 37

My gallery · by **Solloby** » Wed Jun 13, 2018 6:16 pm

It's exciting that progress is being made =D

My gallery · by **Erythro** » Thu Jun 14, 2018 3:15 am

My gallery · by **Lya** » Mon Jun 25, 2018 11:57 pm

So, my head isn't fully working, so please ignore me if this issue has been solved already, buuuuut:

If we still have the issue of having to count certain pets, instead or additionally to narrowing down by date, you can also narrow down using the rarity as all pets relevant to us will be either very rare or omg so rare. That should filter out the biggest part of the pets.
note for daryll: I'd actually use the rarity as the first thing to search for - all pets not having one are either off-account or deleted, plus we won't need any additional data from those, so the program could just skip everything else alltogether. [and I don't think anybody would disagree that we don't need to know if the uninteresting pets are off-account or deleted? it would help with performance and should allow for a bigger range to be processed at once]

edit: don't mind my brain mixing stuff up, but it would still filter out anything deleted or not rare enough to be interesting

My gallery · by **daryll** » Tue Jun 26, 2018 1:49 am

Lya wrote:If we still have the issue of having to count certain pets, instead or additionally to narrowing down by date, you can also narrow down using the rarity as all pets relevant to us will be either very rare or omg so rare. That should filter out the biggest part of the pets.

I looked into this - if you want a different (or modified) program that ONLY looks for certain rarity pets (ie. Very rare) this is extremely do-able.
Found this in a plain-text HTML grab from the program:

Code: Select all: <td class="r"> <img src="/img/rarity/veryrare.png" alt="Very rare"/> </td>

Program can look for specific strings (veryrare, common, uncommon ...) and I can dump the count of those in a txt file:

Code: Select all: Example COMMON: 203 UNCOMMON: 185 RARE: 98 VERY RARE: 37 OMGSR: 3

My gallery · by **Erythro** » Mon Jul 02, 2018 2:11 pm

So far this list does include current list store pets, may remove if we can run the code separately for store pets.

My gallery · by **nickjr** » Mon Jul 02, 2018 4:04 pm

I see the Tiger Rat on the list. What about the black and leopard rats from the same month? Those three were all single-outcome adoptions archive/2008/September/Rat
They're not in the Rarity Changes Archive but I'm not sure if that's because they were more popular or because they just weren't noticed

-squints- I think the list is otherwise good? But there are so many pets lol
Do we have every 2008 pet on there except for the extended releases? The only 2008 non-extended releases that should be missing are those that turned VR after November 2011 (when lil rascal's list ends)

As for store pet ID ranges: Store pets have been released on a pretty regular schedule since 2011ish (maybe 2010), but we might want to cross-reference with old news posts because I feel like there was at least one store pet set after the schedule stabilized that was released about two weeks late and therefore was taken out two weeks later (usually, if the store pets are only released a few days late, they're still taken out on time)

Store pets are archived under the year they were released. So given any Store Pet tab in a year in the Archives, the first set was released in February, the second set was released in April, the third set in June, fourth in August, fifth in October, sixth in December
So to narrow down the ID ranges:
first pet on February 1 through last pet on March 31
first pet on April 1 through last pet on May 31
first pet on June 1 through last pet on July 31
first pet on August 1 through last pet on September 30
first pet on October 1 through last pet on November 30
first pet on December 1 through last pet on January 31 of the following year
(and modify these ranges if needed for the pets that didn't leave on time)

edit: wait... the store pets released until like 2010ish weren't quite released on the same schedule lol I guess you just gotta cross-ref the old news posts to get the date ranges, and if you can't find news posts then... I guess it's time to go forum trawling lol :'D alternatively we could just get... gigantic ranges because that's better than not having a range lol

I'd do it but super busy IRL now

My gallery · by **Erythro** » Mon Jul 02, 2018 4:08 pm

CS Rarity List [WIP] - status update on page 75

Re: Pet Counts for Rarity Ratios - Considering Rereleases

Re: Pet Counts for Rarity Ratios - Considering Rereleases

Re: Pet Counts for Rarity Ratios - Considering Rereleases

Re: Pet Counts for Rarity Ratios - Considering Rereleases

Re: Pet Counts for Rarity Ratios - Considering Rereleases

Re: Pet Counts for Rarity Ratios - Considering Rereleases

Re: Pet Counts for Rarity Ratios - Considering Rereleases

Re: Pet Counts for Rarity Ratios - Considering Rereleases

Re: Pet Counts for Rarity Ratios - Considering Rereleases

Re: Pet Counts for Rarity Ratios - Considering Rereleases

Who is online