What technologies/architectures should I use for this project?

Hi all,

I have to build a platform whit this 2 important requisites:

– Get data from a lot of devices through petitions and receiving CSVs from them every 15 minutes.

– Represent that data (date, and 20-30 variables more) in graphics and to navigate through them.

Many data variables from many devices mean a huge amount of data… and this is the problem.

I have a J2EE project with MySQL database (using EclipseLink) and I save the data in the same table of the database. In the beginning, it was fast but because of the huge amount of data over time, now it is very very slow and the java log shows constant errors (timeouts, etc.).

Thinking in a quick solution, I thought to separate the table where I save the data into various tables depending on the date, for instance, one table for each month and knowing the period date that I have to represent, I could join the data from the different tables I need. That’s how the program could search only the data on a few specific and small tables, not one too big.

I think this solution is a bad practice, but I need something quick to develop.

Another solution I have thought was to use a non-relational database only for that table, but I don’t know how much could speed up the platform.

I ask myself how companies deal with the problem of saving a huge amount of data and keep being fast.

Sorry for my English and my junior ideas!