I think this is a good point to wrap up this learning project. No doubt, there’s still a lot of features that can be implemented, and improvements to be made. However, it was meant to be a training exercise, not a complete production-ready solution. I’ve spent quite a bit of time working on this etude, and achieved some interesting results. I will summarize most significant milestones in this post.

Main achievements throughout the project

I started with a simplified version of the text compression solution, and later upgraded it to a more advanced version that used an adaptive learning algorithm. I also experimented with different substring dictionary limits to see how it impacts the compression efficiency, to come up with a reasonable value.
I had to tackle some performance issues. The first version started with a naive implementation using HashMap; later I replaced it with BTreeMap, which helped performance a bit. However, the algorithm still showed polynomial time complexity, so I implemented a much more suitable trie data structure, and finally got the performance I was satisfied with.
While tackling performance issues, I learned how I could use flamegraphs in Rust to identify the bottlenecks;
On the encoding side, I started with a simple scheme that only allowed me to handle 256 substrings, and later optimized it to be able to encode 2816 substrings. That led to big improvements in the compression ratio.
Along the way, I’ve learned quite a few details about Rust itself, such as: handling multiple crates, differences between HashMap and BTreeMap, details about implementing the comparison traits, and tidbits about the “Rust way” of dealing with error conditions.

Onto the new challenges!

To celebrate the completion of this project, I’m giving the current version a proud number of 1.0.0, and call the job well done!

This project doesn’t end my journey in Rust. I barely scratched the surface, and there are many more exciting discoveries waiting for me. I look forward to a new challenge that hopefully will let me dive deeper into subjects like:

multi-threaded programming and asynchronous Rust;
working with networks;
building user interface applications, both CLI and GUI.

That’s all, folks! Thanks for your attention, and see you next time!