XML2CSV

XML to CSV Documentation

v01.01.07

Released: 09/27/08

Program Summary

Using the Fairlight XML to CSV Converter, you can generate straightforward CSV files that let you properly trace the flow of data in the original heirarchical structure, while eliminating the cumbersome parsing inherent in the design of XML. This tool is an ideal companion for applications that don't inherently understand XML, but which import CSV very easily.

This command-line based program may be used directly, or used in automated systems. It is possible to build a GUI or web-based wrapper around the program (see the demo page below for a live example of such a creature).

Supported Platforms

Currently supported platforms are:

Windows 9x/NT/ME/2000/XP/2003
Any other platform that supports Perl and the required modules listed below.

Required Modules

The program requires the following modules be installed for Perl on any non-Windows platform:

XML::Parser
MIME::Base64

The self-extracting executable version for Windows meets all of these requirements for you. That version does not require perl be installed at all.

Demo Versions

This program can be tested via a web front-end designed specifically for this purpose.

Click here to demo!

Installation

Windows

Run the Win32 installer.

Other Platforms

Unpack the compressed tar or zip archive into any directory you choose. A subdirectory named xml2csv-install will be created, and all program and documentation files are present within this subdirectory. You may relocate the program from this directory to elsewhere.

Edit the first line of the program file to reflect the location of your perl binary!

Usage

xml2csv [options] xml_filename

Options

--config, -c file

This option dictates an optional configuration file for the conversion.

--outfile=file, -o file

This option denotes to what file the CSV output is written.

--strip, -s

This option strips node paths from element name in the CSV file. Only the basename of the element will be provided if this option is used.

--fieldsep=character, -f character

This option lets you change the comma separator to some other arbitrary character (or characters).

--quote=character, -q character

This option lets you change the double-quote character to some other arbitrary character (or characters). An example would be a single-quote.

--working-dir=path, -w path

This specifies the path to which any decoded Base64 files will be saved.

--debug, -d

This option is for internal xml2csv development, and/or bug report traces. This option should not normally be used unless you enjoy creating large files for no reason.

--version, -v

This option returns the program version information.

--changelog

This option returns the program changelog information.

--help, -h

This option returns the program help (a briefer summary of the options) in a self-paging fashion.

Configuration Files

Optional configuration files may be used with XML files to handle two specific tasks via the -c option. The first task is the decoding and saving of Base64 encoded data in decoded form to a separate file. Within configuration files, you will specify an element or an attribute's node path. This is the list of all heirarchy elements up to the element or attiribute you intend to affect, including that element or attribute, in a single string separated by periods. Configuration lines for the functionality described above are in the format:

B64:node.path.to.encoded.element

If that syntax is used, the decoded file will be written to the working directory (see the --working-dir option), and the generated safe filename will be injected into the CSV data stream as the field value for that node.

The second task is the exclusion of nodes or whole sections of nodes. The format of the rule is:

E:node.path.to.excluded.element

If this rule is used, the element indicated, all attributes, and all child elements will be excluded from the output. Please note that the syntax is identical for excluding an attribute of an element and an actual element. If you simply wish to exclude attribute size of element field_submission.form_field, you would specify E:field_submission.form_field.size as the rule.

All rules should be entered one rule per line. Blank lines and lines beginning with a pound sign (#) will be ignored. If you do not need to extract Base64 encoded data, nor exclude nodes, you do not need to use a configuration file.

Results

The results of XML to CSV conversion with this tool yield a CSV file made up of two fields per line, in the format:

"field1","field2"

Lines that are straight element or attribute values will generate lines where field 1 is the element or attribute name (including the node path, if --strip is not used), and field 2 is the value of that element or attribute. The following would be an example of such a line:

"BatchEntries.BatchEntry.BatchCreatedBy","Mark Luljak"

Additionally, you will receive indicator lines that point to the status of progression through the data heirarchy. Any element that has no character data will generate the following type of line when that element opens. The line has a static first field indicating an element opened, and the second field will indicate which field was opened, like so:

"__FLT_TAG_OPEN__","fieldname"

Likewise, when the end of such an element (one with no character data) is encountered, the following line type will be generated. Again you will see a static first field, this time indicating that an element has closed, and the second field will show you which element closed, as illustrated below:

"__FLT_TAG_CLOSE__","fieldname"

Elements with or without character data can still have attributes. In all cases, attributes are displayed with lines identical to the format used for data-containing elements. In the case of an empty element, you will receive an opening indictor line, a line for each attribute (if any are present) for that element, and then a closing indicator line.

Indicator lines for open and close of element are meant to help you track your progress through the data relations in the XML.

Error Detection

All errors will be written to standard output. This means that if the program prints anything to what would be the terminal (or the console window in Windows), there was an error. Redirection of output to a file in any operating system will let you see if there was an error. The error will be contained in the file. If the program was 100% successful, there will be no output of this type, and any redirection destination file would be zero bytes long. If you use this method to catch errors in an automated system, check the file size after using a redirect with the xml2csv command. If the file is larger than zero bytes, check the file for the exact error.

Changelog

v01.01.00 - 12/30/07

Original Release.

v01.01.03 - 01/05/08

Fixed missing field open/close tags on self-closing fields. Fixed concurrent same-name element entries not performing as expected. Fixed double-internal-end blank+newline issue. Fixed (introduced) bug that didn't delete character data in one tag close condition.

V01.01.04 - 09/19/08

Fixed blank line issue if all attributes for a tag were excluded in configuration file.

V01.01.07 - 09/27/08

Fixed SUID taint issues for filePro SYSTEM() functionality.

This software remains the property of Fairlight Consulting, and license to use the program is sold to users. Fairlight XML to CSV Converter is licensed at a cost of $995.00 USD per server on which it is installed. Each license fee entitles you to use the program on one server, in any role you require.

Upon receipt of payment for a license, program will be delivered to the licensee via email at the email address associated with the PayPal payment.

Upgrades for the product are free when moving to new minor and bugfix point-release versions. Major version upgrades will be paid upgrades. Fairlight Consulting reserves the right to change this policy in the future, with no prior warning.

There is no warranty for this software. This software is offered "AS-IS" and without warranties as to performance or merchantability or any other warranties, whether expressed or implied.

Good computing practice dictates that any program should be thoroughly tested with non-critical data before deploying it for production. The user assumes the entire risk of using the program. In no event shall Fairlight Consulting be held liable for loss of data, failure of performance, or any other damages, be they real or perceived.