[Week 10] Overthrowing the King of Bugs

In the end, all it took was a few nights, several scribbled whiteboards and a ton of manually going through 9000 line long JSON files.

Summary

• Fixed handling of split chunks in IndexStorage.update_lookup_table 
• Began working on performance evaluation
  

Details

The stuff in the intro is true. It did take a lot of work to figure out exactly how to best fix the split chunk handling bug. I started by logically working through every edge case and narrowing down the issue to the handling of "fraction of bucket" files. Then began experimentation on methods to fix the problem.

I'll spare you the gory details, but trust me when I say that the number of lines changed in the fixing PR does not indicate the amount of debugging code and supplementary code I had to write in order to analyze this issue.

I've never mentioned this before, but generating data for the tests is not an easy job at all. In fact, the amount of code I've written just to generate the test data rivals the size of the main library itself!

Anyway, once that was done, I started working on performance evaluation, which is one of the last milestones remaining for this project.