ETLyte - CSV Validation

Hi all,

 

Not sure if this is the right place to pass along the info, but I wanted to
let people know about an open source tool I've been working on, ETLyte.  I
was inspired by CSVLint, but needed the ability store/output errors to disk
(not in memory) and handle multiple, linked, large files.

 

Its name certainly speaks to its growing ability to handle ETL tasks, but
what I originally built it for is cross-file validation.  I have a client
who receives 10 separate files linked by a ContractID in each, so we have
used the tool cross-validate.  Best of all, because it populates a SQLite
database, all of the custom validations can be written in SQL, and even
executed ad-hoc after a "run."  Flatfiles are described using a similar
format as CSVLint (working on making them compatible) and datatypes are
enforced with built-in .NET functions, and there are many other tools
built-in to help non-programmers wrangle CSV data.

 

It's very much a work in progress, but can be found here:
https://github.com/sorrell/etlyte with more thorough documentation/examples
here: https://sorrell.github.io/etlyte  

 

--

Nick Sorrell

 <http://cint.io/nick_sorrell.pub> PGP Public Key

513.288.1084

 <https://cint.io> https://cint.io