Table Finder
Some of you have noticed that it seems like it is taking a very long time for hands to show up on PTR. You might also have noticed that the top winners/biggest losers lists are not updating lately. There is a very good reason for this. PTR is constantly adding hardware to support our growing database, in fact we add several new database nodes a month to accommodate our roughly 2.5 TB per month of new data! If you care to know, the database already spans more than 20 servers and has over 100 billion rows.
Some of you have noticed that it seems like it is taking a very long time for hands to show up on PTR. You might also have noticed that the top winners/biggest losers lists are not updating lately. There is a very good reason for this. PTR is constantly adding hardware to support our growing database, in fact we add several new database nodes a month to accommodate our roughly 2.5 TB per month of new data! If you care to know, the database already spans more than 20 servers and has over 100 billion rows. What happens, in general terms, is that without new nodes to handle the growing data set – the old nodes start to get slower and slower, and eventually they can no longer keep up with the incoming volume (which is now in the neighborhood of 18 million hands per day). When these nodes start to slow down, a backlog of incoming hands starts to build up. You can see this when checking your PTR after a session and cannot find your latest stats, but they appear the next day – this is from a parsing backlog.
A failed experiment of ours in server technology caused an interruption in our supply of fresh database nodes in late March, so we had to emergency order new database nodes. Those nodes arrived and were installed, configured and brought online yesterday. We’ve now started to catch up on the backlog – but on PokerStars alone we have a backlog of approximately 23 million hands that’ll need to be parsed in addition to new hands which never stop. We hope to catch up with this backlog within a week.
The top winners feature was disabled in hopes of gaining performance on the saturated nodes, to allow them to keep up with the incoming data. This clearly didn’t work. This feature will be re-enabled when we are satisfied that things are stabilizing again.
The bottom line is that no hands have been lost, and we aren’t getting lower than usual coverage – but there is a longer than usual delay in getting those hands parsed and into our database. In the mean time we’re working to evolve our database technology to handle the ever growing dataset, and serve you better.
Thanks for understanding and being patient while we work through this!
Dameon
Tags: Technical Update
2
2
Ever considered using something like Cassandra and AWS?
Please post pic of servers for fun
@chantech: We have but there are a few factors, first we’ve already got a 25TB infrastructure in place. Also we’re not comfortable with the maturity of these projects just yet. Distributed Hash Tables show a lot of promise but until we see more real world usage that doesn’t require extensive development (like Digg, facebook, etc do) – we’ll let others guinea pig it ;)
Specifically which AWS are you referring to? We have tested a variety of these out, and much of our static content is actually served by cloudfront/S3. The pricing model of EC2 is still prohibitive for us at this point.
@johan313: Actually the picture for this article is a shot of one of our servers, if you look closely you can see there is a failed drive in the pic too ;)
Comment Removed.
Mod note: Please refer to commenting policy @ http://www.pokertableratings.com/commenting.php
Comment Removed. Comment ban in place.
Mod note: Please refer to commenting policy @ http://www.pokertableratings.com/commenting.php
good stuff PTR, appreciate the good work, thanks!
Do the new table labels on Stars have any effect on the coverage?
Will the ‘My Recent Searches’ feature be back?
The new tables should not effect coverage.
Yes the recent searches feature will be back soon :)
Silence! I have something to say
saying no hands have been lost is a lie. You’ve lost a couple thousand of my hands in the last 3 weeks. And no, I’m not sending you the hand histories PTR since you refuse to add them to your database and fix my PTR page
Out of curiosity (I am a SW eng myself) – what DB solution do you use (MySQL, MS-SQL, Oracle etc’)? If it’s not a trade secret of course
I Just want to tank you for this website, I think it so cool that you make it accessible to everybody at no cost, I’m very fortunate to be part of this group . Congrats on the work !!!!!!
“saying no hands have been lost is a lie. You’ve lost a couple thousand of my hands in the last 3 weeks. And no, I’m not sending you the hand histories PTR since you refuse to add them to your database and fix my PTR page”
What’s the point of this? You’re telling them that something is wrong, but refusing to help resolve it.
LOL at 4th an 1….do you really think PTR care if you don’t send your hand history to them?
Just be happy they are providing this service. Don’t like it?….bye then.
There is a thread in our forum in which we address the issue of missed hands, and explain why we ask for hands but do not add them to the database. You can check it out here: http://www.pokertableratings.com/forum/general-ptr/missing-hands-read-this-t54.html
@armor32: Highly customized version of mysql.
@dpereiraf: Thanks :)
Good work, Dameon, regards Engin
really appreciate the work u guys do, keep it up.
are u ever gonna track hands that are run twice? my HEM doesnt have a problem tracking them. thx
thanks for the update, you guys rock!
@4th an 1. Are you for real? Did you read what was just written in this article? 2.5TB per month of data, 18 million hands a day, and you complain because they miss some of your hands lol, get over your self. Your own software can track your hands and this site is a free bonus for everyone to use for lots of other useful information. I and I’m sure the majority of other users appreciate the hard work that these people put in to make this site the best poker data site online. I really can’t believe you posted that, it’s not compulsory to use the site.
@25lighterz: We plan on it long term but it is low priority at the moment.
Thanks for the kind words guys!
armor32 says:
April 15, 2010 at 9:53 pm
Out of curiosity (I am a SW eng myself) – what DB solution do you use (MySQL, MS-SQL, Oracle etc’)? If it’s not a trade secret of course
I had the same question after reading how HUGE your data base is. If you are comfortable providing additional info, I would love to read it. Thanks.
@ all the haters – blah blah blah you’re all dirty tramps! Agreed, this site is an amazing free tool.
I would like to say how fantastic this site, not only is it free, i was also very grateful to win a $75 gift voucher ( i bought more poker books with it ) for picking a correct hand in a “free competition” , some people just dont get it – 4th and 1. You are a sad person.
Thanks heaps Dameon for always keeping us updated, and for also providing the best poker site on the net.
Good stuff and thanks for giving some insights in why the delays are occuring. Keep up the good work and thanks for offering an amazing tool
I just pray everything will go fine for you in the future, very good site guys!
I want to thank you for your amazing work, this is an exceptional tool. The best side of course, and free – FANTASTIC.
I couldn´t make my own stats, and here i see mine and from the hole comunity.
Thank you for the explanation and keep it up.
best regards – cristiano
I don’t know you, but I like you !!!
from Russia with love, GuestZ.
My appreciation! – Thanks much guys for your effort & excellent site.
@cjr123: Quite a few people have been asking about our technology. We might in the future write a bit more about what powers the site, to the extent that we can reveal ;)
Best thing you can do to support the site is to order hands in the limit you play and incorporate them to your pokertracker database, I´m starting to eliminate the leaks in my game trough replaying hands from the top winners of the limit, watching their starting hands and position hoping to develop a winning strategy myself.
Add a good colection of articles about strategies to win in microlevels and a participative forum and you´ll get the best poker site around.
Keep the good work! And greetings from Andorra.
How/Where does ptr make its money?cause I know the technology you guys use is not very cheap,is it all just with advertising the poker sites?
@jpd i’m not sad baby, I’m living the dream!
“How/Where does ptr make its money?cause I know the technology you guys use is not very cheap,is it all just with advertising the poker sites?”
Selling hand histories.
Thanks for all your hard work.
Its still too laggy
Fantasic Site and really great work maintaining all the data….. problems will also occur but PTR will always Triumph in the end…!!
Keep it up and Ignore the negative comments..!!
Damon:
First: Great your hardware gets more powerfull… It is needed… Thanks for the effort…
Second: “The top winners feature was disabled in hopes of gaining performance”
Not nice, but ok when needed… I really like this feature… May we get those missing daily, weekly and monthly top-stats back? I mean… May you install an archive for that? Or something like: “Show me the top winners/losers, PLO100, Party, from April 4th to April 14th or something like that? If you will show an archive-page of weekly-winners this should not be that performance-hitting as this archive-page could be static without database-querys… But this would be a huge improvement! :)
I would even pay for such an archive-list… when it stops the need to wake up on a sunday morning to get the most actual weekly-stats. :-)
Or you could do this winner/loser-stats completely static in an archive without any on-the-fly database-querys… I am just interested in looking for an archive from last week/last month and not from this week/month as it is mostly not complete besides on the last day…
“@chantech: We have but there are a few factors, first we’ve already got a 25TB infrastructure in place. Also we’re not comfortable with the maturity of these projects just yet. Distributed Hash Tables show a lot of promise but until we see more real world usage that doesn’t require extensive development (like Digg, facebook, etc do) – we’ll let others guinea pig it ;)
Specifically which AWS are you referring to? We have tested a variety of these out, and much of our static content is actually served by cloudfront/S3. The pricing model of EC2 is still prohibitive for us at this point.”
Interesting…in my experience with Cassandra its been pretty stable. It didn’t used to be that way, and for awhile FB open sourced it and then didn’t really support or apply any patches to it and I thought it wasn’t going to be supported. I was just wondering because the site seems slow sometimes and I think a lot of that is probably a MySQL bottleneck.
I meant EC2 when I meant AWS. It seems like you wouldn’t have to wait for servers to arrive, configure them, get them deployed, etc.
By the way, that’s a sick amount of data you guys are working with…
Top winner/loser come back, but sorting don’t work…
FMS PDF Metadata Batch Editor is an easy-to-use tool to modify pdf files XMP metadata….
[...]PTR Technical Update | PokerTableRatings.com Blog[...]…