Skip to main content

Validating CSV Files

What is CsvValidator ?
  A Java framework which validates any CSV files something similar to XML validation using XSD.

Why should I use this ?
  You don't have to use this and in fact its easy to write something your own and also checkout its source code for reference.

Why did I write this ?
  Some of our projects integrate with third party application which exchanges information in CSV files so I thought of writing a generic validator which can be hooked in multiple projects or can be used by QA for integration testing.

What is the license clause ?
 GNU GPL v2

Are there any JUnit test cases for me checkout ?
 Yes, source

How to integrate in my existing project ?

Just add the Jar which can be downloaded from here CsvValidator.jar and you are good.

Instantiate CsvValidator constructor which takes these 3 arguements

         // filename is the the file to be validated and here is a sample
        // list - defines all the fields in the above csv file ( a field has index, type, isOptional, regex )
        // last argument is the file delimiter and it can be anything and not just comma


Checkout this sample code 


 public static void main(String[] args) {
        boolean optional = true;
        boolean notOptional = false;

        List list = new ArrayList();
     
        list.add(new Field(1, Type.NUMBER, notOptional));
        list.add(new Field(2, Type.NUMBER, notOptional));
        list.add(new Field(3, Type.TEXT, notOptional));
        list.add(new Field(4, Type.TEXT, notOptional));
        list.add(new Field(5, Type.NUMBER, notOptional));
        list.add(new Field(6, "Purchase Date", Type.DATE, notOptional, "yyyy-MM-dd HH:mm:ss"));
        list.add(new Field(7, Type.NUMBER, notOptional));
        list.add(new Field(8, Type.NUMBER, notOptional));
        list.add(new Field(9, "Campaign ID", Type.TEXT, optional));
        list.add(new Field(10, "Promo Code", Type.TEXT, optional));
        list.add(new Field(11, Type.TEXT, notOptional));
        list.add(new Field(12, Type.NUMBER, optional));
        list.add(new Field(13, Type.NUMBER, optional));
        list.add(new Field(14, Type.TEXT, notOptional));

         
        CsvValidator validator1 = new CsvValidatorImpl("somefile.txt", list, "\\|");


        if (!validator1.isValid()) {
            System.out.println(validator1.getValidationDetails());
        }

    }

Can my QA use it in a stand alone mode ?
 Yes, just checkout the spec.txt file which your QA needs to create.

java -jar "CsvValidator.jar" csv-file.txt spec.txt

first line in spec.txt is the delimiter (, ' | etc)
  All other lines contains each field information on a separate line for example
 
    Currency,T,R,
    Date of Purchase,D,R,yyyy-MM-dd HH:mm:ss

    first column - field name helps in understanding validation results 
    second column - type (T, N, D ) represents Text, Number, Date
    third column - Required (R, O) represents Required or Optional
    fourth column - regex 

  (Regex for dates )
    
"yyyy.MM.dd G 'at' HH:mm:ss z"2001.07.04 AD at 12:08:56 PDT
"EEE, MMM d, ''yy"Wed, Jul 4, '01
"h:mm a"12:08 PM
"hh 'o''clock' a, zzzz"12 o'clock PM, Pacific Daylight Time
"K:mm a, z"0:08 PM, PDT
"yyyyy.MMMMM.dd GGG hh:mm aaa"02001.July.04 AD 12:08 PM
"EEE, d MMM yyyy HH:mm:ss Z"Wed, 4 Jul 2001 12:08:56 -0700
"yyMMddHHmmssZ"010704120856-0700
"yyyy-MM-dd'T'HH:mm:ss.SSSZ"2001-07-04T12:08:56.235-0700

 Regex cheat sheet for all others cheat sheet

Comments

Anonymous said…
Excellent article and easy to understand explanation. How do I go about getting permission to post part of the article in my upcoming news letter? Giving proper credit to you the author and link to the site would not be a problem.
Anonymous said…
In my csv some headers are created dynamically, i.e they are unknown at compile time. Is there any way to validate those field using CsvValidator?
vijay krishna said…
Can you share gradle build path for this jar
Unknown said…
Thank you a lot for providing individuals with a very spectacular possibility to read critical reviews from this site.
Java training in Chennai

Java training in Bangalore
Mounika said…
The knowledge of technology you have been sharing thorough this post is very much helpful to develop new idea. here by i also want to share this.
python training in chennai
python course in chennai
python training in bangalore
Diana Ayu said…
AGEN TOGEL ONLINE TERPERCAYA CERIA4D

WA : +62 82393544066

#PROMO CERIA

-BONUS DEPO 10RB

-BONUS NEXT DEPO 1%

-BONUS REFFERAL 2%

-JACKPOT 4D 3D 2D NO HP TOTAL HADIAH 5 JUTA !!

-LOMBA GRUP HADIAH RATUSAN RIBU !!

-HADIAH PRIZE 2 DAN PRICE 3 !!! << NEW !!

- TERSEDIA DEPOSIT VIA " PULSA "

- LIVE GAME CASINO & SBOBET (BOLA)

- Fitur Game Casino (POKER , SBO )

- BEBAS LINE 2D/3D/4D

_________________________________________

DISCOUNT TOGEL dibawah ini :

>2D DISCOUNT : 29% x 70

>3D DISCOUNT : 59% x 400

>4D DISCOUNT : 66% x 3.000


LINK DAFTAR:
WWW . JPCERIA . COM

CERIA
BANDAR TOGEL ONLINE
TOGEL ONLINE
JUDI ONLINE
BANDAR ONLINE
BANDAR TOGEL
AGEN TOGEL ONLINE
MAIN TOGEL ONLINE
BANDAR TOGEL AMAN
MENANG TOGEL
BERITA TERBARU
KODE ALAM
TOGEL
CERITA DEWASA
DEWASA
Shivam Kumar said…
well explained . thanks for sharing good post . Good arraylist example collection visit
Arraylist example

Popular posts from this blog

Access multiple Databases in JPA

According to JPA specification we can define multiple "persistence-unit" elements (i.e. like below) in persistence.xml file and can easily refer them inside Dao layers as this. public class PolarDaoImpl {     @PersistenceContext(unitName="PolarPU")     protected EntityManager entityManager; -- } public class BearDaoImpl {     @PersistenceContext(unitName="BearPU")     protected EntityManager entityManager; -- } Checkout sample persistence.xml <?xml version="1.0" encoding="UTF-8"?> <persistence version="2.0" xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">     <!-- Database 1 -->     <persistence-unit name="PolarPU" transaction-type="RESOURCE_LOCAL">         <

JPA 2 new feature @ElementCollection explained

@ElementCollection is new annotation introduced in JPA 2.0, This will help us get rid of One-Many and Many-One shitty syntax. Example 1: Stores list of Strings in an Entity @Entity public class Users implements Serializable {     private static final long serialVersionUID = 1L;     @Id     @GeneratedValue(strategy = GenerationType.AUTO)     private Long id;     @ElementCollection     private List<String> certifications = new ArrayList <String> ();     public Long getId() {         return id;     }     public void setId(Long id) {         this.id = id;     }     public List <String> getCertifications() {         return certifications;     }     public void setCertifications(List <String> certifications) {         this.certifications = certifications;     } .. }         Users u = new Users();         u.getCertifications().add("Sun Certified Java Programmer");         em.persist(u); Generated Tables    Users    Co

Reuse JPA Entities as DTO

Note : Major design advantages of JPA Entities are they can detached and used across tiers and networks and later can by merged. Checkout this new way of querying entities in JPA 2.0 String ql = " SELECT new prepclass2.Employee (e.firstname, e.lastname) FROM Employee e "; List<Employee> dtos = em.createQuery(ql).getResultList(); The above query loads all Employee entities but with subset of data i.e. firstname, lastname. Employee entity looks like this. @Entity @Table(name="emp") public class Employee implements Serializable {     private static final long serialVersionUID = 1L;     @Id     @GeneratedValue(strategy = GenerationType.AUTO)     private Long id;     @Column     private String firstname;     @Column     private String lastname;     @Column     private String username;     @Column     private String street;     @Column     private String city;     @Column     private String state;     @Column     private String zipc