Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Not strictly today. But I discovered that there exists a special class of algorithms that are designed with the use case of streaming data to your program. I just used one to get a uniformly distributed sample from a 10Gb log file.

I knew this was something coding interviews delved into: "if it doesn't fit in memory", but until like yesterday I never went down the rabbit hole. I have to say it was a nifty trick.






Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: