I'd like to understand a bit more about your data structures. Raw text should live in a file system, but if you're creating highly structured data using a SQL database could be a good fit. I'd also suggest taking a look at Spark; you can run it locally on your machine (does not need a Hadoop cluster) and can interact with flat files. If you're finding that your machine is lacking in horsepower or space, AWS could be your friend.