- From: Ceolin, D. <d.ceolin@vu.nl>
- Date: Fri, 21 Mar 2014 16:56:43 +0000
- To: "public-csv-wg@w3.org Working Group" <public-csv-wg@w3.org>
Received on Friday, 21 March 2014 16:57:12 UTC
Hi all, I think that this post I've just stumbled upon is relevant for our work: http://www.r-bloggers.com/it-is-time-for-rdata-files-to-become-the-standard-for-data-transfer/ in particular, it says: "I was recently attempting to download weather data from the US government and found myself stymied because the dataset I wanted was considered too large (+5 gigs). The problem I realized was not that I wanted too much data but that the data transfer format was so poor. The only datasheet form available was csv. I was therefore forced to drop many variables and resubmit the data request. Ultimately, I downloaded only some of the pieces of the data which ended up being a file 627 MB in size. Importing the data into R via the read.csv command and immediately saving it as an “RData” file reduced the size to 55.3 MB (92% reduction). " I've attempted to touch on similar issues in one of the use cases I've proposed, and I think there might be work to be done in this direction for the "CSV to Tabular Data Framework" conversion. Cheers, Davide
Received on Friday, 21 March 2014 16:57:12 UTC