FILECONVERT Version 3.0 - Documentation
Copyright (C) 2007.2008 by Adrian Cornwell
All Rights Reserved
Introduction
Fileconvert is a command line based tool designed to read an input data file and
by applying a template file outputs the data in a different format.
For Personal Use this software is Free, but if you are using it in a company or on
a corporate network then a shareware License IS required.
For example, you could convert a basic data file (fixed length or CSV) to html or xml,
or an xml file to csv or even convert DOS files to unix format and visa-versa,
you are limited only by your imagination.
The basic process flow of fileconvert is ….
1. Read Template File to decide how to interprete the input file and what output is required.
2. Read the input file and process based on information gathered from step 1 and produce the output.
Our Example Application quickconvert (included) provides some prewritten examples to perform some common tasks
and generate simple templates to get you started (see the Quick Start section below)
Please check our website www.dev2go.com from time to time for upgrades and also for
real-life applications of this program.
Quick Start
The best way to get up and running and familier with fileconvert is to install it (see next section) and then run the
Quickconvert.bat program. This is a simple DOS menu that provides access to some common functions that you might wish to
perform and can also be used to generate some basic templates to get you started. This application is discussed in more detail
in the Example Applications section of this document, but basically it provides the following options:
- Convert File(S) from Unix to DOS Format
- Convert File(S) from DOS to Unix Format
- Perform a Search and Replace
- Perform a Search
- Convert a CSV File and generate a Template
License Agreement
You should carefully read the following terms and conditions
before using this software. Unless you have a different license
agreement signed by Adrian Cornwell your use of this
software indicates your acceptance of this license agreement and
warranty.
This software is free for personal use only.
If used by a company or on a corporate network then a shareware license MUST be purchased
(see www.dev2go.com for details).
Additionally, any form of reverse engineering of the fileconvert program is forbidden.
Disclaimer of Warranty
======================
THIS SOFTWARE AND THE ACCOMPANYING FILES ARE SOLD "AS IS" AND
WITHOUT WARRANTIES AS TO PERFORMANCE OR MERCHANTABILITY OR ANY
OTHER WARRANTIES WHETHER EXPRESSED OR IMPLIED.
Installing/Upgrading fileconvert
Simply unzip the downloaded fileconvert.zip into a directory of your choice.
The directory will contain the fileconvert.exe program and this documentation plus a templates and docs sub directory.
This documentation assumes that you have installed the program into a directory called c:\fileconvert.
If you have installed it into a different directory then you will need to edit the quickconvert.bat file and change
the SET FDIR parameter to the directory you are using (this is described in the bat file itself).
An extract of the file is shown below with the relevent lines in bold text.
echo off
rem module: quickconvert.bat
rem Author: Adrian Cornwell @ www.dev2go.com
rem Description:
rem This DOS program provides a simple interface for performing
rem various conversion and replace functions using a simple
rem dos menu.
rem ======================================================================
rem Configuration
rem ======================================================================
rem Depending on the directory in which you installed fileconvert you
rem may need to change the parameters below
rem Set the parameter below to the directory in which you have installed
rem fileconvert
SET FCDIR=c:\fileconvert
rem Set the parameter below to the directory in which you would like
rem temporary files created.
SET FTEMP=c:\temp
rem ======================================================================
rem Display Welcome and start up
rem ======================================================================
Update History
Version |
Description |
3.0 |
A Major Release containing the following;
- Enhancement Memory Management. The -M and MEMBLOCK Commands are now depreciated as no longer needed, the
program will handle all memory internally.
- When the MAKE-UPPERCASE and MAKE-LOWERCASE are used before Field Defintions they will be applied only
to the input stream and not to the output so your output section will remain unchanged.
- You can now use EOF as a Field Delimiter to read all data to the end of the file.
- Added new Template Command SET-EMBEDDED-STRINGS.
- Removed -AC, -AH and -AX command line options as templates can now be generated using the examples.
- -OINPUTFILE Command line option added
- You can now use -F@file to import Filter Commands
- By Default, Filters are case sensitive. You can make a filter case insensitive by using the new FILTER-IGNORE-CASE
template command or the new -FC command line option
- New command TRIM-MULTIPLE-SPACES added
- New options SORT and SORTDESC added to START-OUTPUT Command and also -SA -SD Command line options added.
|
2.3 |
Added SPECIAL Command for filtering Special Characters when reading binary Files.
Added @ option to -I command line parameter allowing a file containing a list of input files to be passed.
Added EXCLUDE-FILTER Command to revserse Filter Criteria.
Added TRANSLATE-INPUT Command to parse input stream.
|
2.2 |
You can now use -F runtime Filter Commands as often as needed (was previously limited to 5).
You can now use -F@file runtime filter to read a list of filters from a file.
FILTER Command has been added.
UNIQUE option added to START-OUTPUT Command.
|
2.1 |
Fixed bug in multi-page reports that produced wrong headers/footers when only one page produced.
Removed Aggregate Functions as not really a file conversion issue.
Output File now optional. If missing output will be directed to screen
Added {{INPUT-FILENAME}} and {{INPUT-SHORTNAME}} Markers
Added -AC, -AH and -AX command line options to allow auto generation of Templates.
Added -F command line option to auto runtime filters and also added {{FILTER}} Marker.
Added several utility programs into the examples directory
|
2.0 |
Major extensions in this version. old version syntax remains unchanged but additional tags and commands have been added to support multi-page outputs
|
1.4 |
Added COPY-FIELD Command
Added DEFAULT-P1 to DEFAULT-P5 and -P1 to -P5 Command line parameters
Added {STARTPOS..},{ENDPOS..} and {LENGTH..} Tags
Added -B and -S command line parameters
|
1.3 |
Added COUNT-IFNUM & COUNT-IFNOTNUM Aggregate Functions
|
1.2 |
Added MAKE-UPPERCASE & MAKE-LOWERCASE Commands
Added Aggregate Functions to Footer Section
|
1.1 |
Added {:asc} marker for special characters
Improved Memory freeups when template validation fails
File reading uses read buffer to improve performance
Added -R Command line parameter
|
1.0 |
Initial Release |
Contact Us
If you have any questions, suggestions or feedback please use the contact us form on our
website (www.dev2go.com).
Sending an email directly may not
get read due to the amount of spam we receive, but using the contact form will always be read.
Basic Command line parameters
The basic parameters are the input file, the template file and the output file which are specified as shown below.
-I"filename"
| Specifies the input filename.
You can also use -I@"filename" if the filename contains a list of files to be processed. This allows
multiple input files to be processed into a single report.
|
-T"filename"
| Specifies the template filename
|
-OINPUTFILE
| Specifies the output filename.
Specifies that the Output File is the same as the input file.
|
-O"filename"
| Specifies the output filename.
If not passed then output will be directed to the screen, If
you do this then I suggest that you use the -S (Silent) mode flag to prevent normal
fileconvert prompts being displayed within your screen output.
Example:
This example reads input file based on template and outputs the results to the screen which is then
redirected to a file (DOS Syntax)
fileconvert -Imydata.dat -Tmydata.txt -S >c:\temp\output.txt
|
Example:
fileconvert -Imydata.dat -Tmydata.txt -Omyrep.htm
The example above will result in fileconvert reading the template file mydata.txt and then applying this logic to the input file mydata.dat to produce the output file myrep:htm.
Advanced Command line parameters
This section describes additional command line options for advanced users
-U
| Specifies that Template File is in Unix Format
|
-V
| Specifies Verbose Mode. All processing of the input file will be displayed on screen.
|
-S
| Specifies Silent Mode. only errors and warnings will be displayed on screen. Note that this parameter is ignored if you are running with the -V flag option
|
-B
| This turns on Buffered output.
The input file will be read and the output will be saved in memory. after the input file is closed then output will be written. This is a useful option if your input and output files are the same filename. Note that you cannot use this option if your template produces multi-page output.
Note that when running in this mode you can only output a maximum of 1 Million Records.
|
-Fstring
|
Allows you to specify a runtime filter to be applied to the file conversion meaning that only records containing
the search string will be outputted (case is ignored).
You can pass this parameter as many times as required .
Also see the {{FILTER}} marker in the Template Commands - Output Section of this document.
Example:
-F"test1" -F"test2"
Only records whose data contains the strings test1 OR test2 will be printed.
If you wish to use a large number of filters, then you can simply list them in a text file and use
the following syntax to read the filters directly from this external file. -F@filename
Example:
Assulming file c:\temp\filter.txt contains the following..
test1
test2
test3
You could include the file using
-F@c:\temp\filter.txt
Note that non-printable characters can also be used using the {:dec} markers (see Special Characters section for more information).
The example below will search for data containing TAB Character.
-F"{:9}"
As a Final note if using an external file, it can also contain full filter commands as described in the templates input section of this document (FILTER string DISPLAY string).
Also refer to the FILTER Command that can be used within your template
|
-FC
|
Will force all filters to use a case-insenstive search (see FILTER-IGNORE-CASE command for more info).
Example:
-FC
all filters will ignore case when searching.
|
-Rrecs
|
Will display a message on screen whenever recs number of records are processed.
Example:
-R100
to display a message every 100 Records.
|
-Pn"Value"
|
Allows you to pass upto 5 runtime parameters (-P1 to -P5) directly into your template.
Example:
-P1"some text"
Will define the template marker {P1} as some text
|
-SA
|
Will sort the report output into Ascending Order (Note that this cannot be used for multi-page reports)
|
-SD
|
Will sort the report output into Descending Order (Note that this cannot be used for multi-page reports)
|
Template Basics
The template files are the engine of the program and tell fileconvert both about the input file and what
output is required.
A template contains simple fileconvert commands and can be written using a basic text editor.
templates are devided into 2 sections. everything above the mandatory START-OUTPUT command
defines the structure of the input data and any actions that need to be performed on it and
everything below this section describes the output that will be produced.
This section does not describe the commands in detail as it is meant provide a basic understanding of
the concepts used.
The first step in building a template file is to understand the FIELD Commands. It is these commands that
are used to structure the input.
If for example, you are reading a semi-colon seperated file (example below)
Customer_Number;Customer_Name;Location
001;Joe Bloggs;London;
002;John Smith;Paris
Then your template could contain the field structure below..
FIELD CUSTNUM ";"
FIELD NAME ";"
FIELD LOC "{:13}{:10}"
Note that the Template Commands and Field Names should always be upper case.
In the above example fileconvert will put everything before the first semi-colon into the field CUSTNUM,
then everything upto the next semi-colon into the field NAME and everything upto the next carriage-return+linefeed
into the LOC field. This loop would continue until the end of file is reached.
Note the special {:13}{:10} syntax on the last field. This is how fileconvert can handle special characters.
The basic syntax is {:dec} where contains the ASCII Code of the character. Some common ones being 9 (Tab),
13 (carriage-return) and 10 (linefeed). Windows and DOS based files use a Carriage-return+linefeed as an end of record
marker, but Unix uses only linefeeds, so if you were reading unix files you would delete the {:13} marker.
If you you wished to make the template file type inderpendant, you could use the following syntax for the last field.
FIELD LOC "{:10}"
REPLACE "{:13}"
in this approach, the LOC field would contain everything upto the linefeed (including the carriage-return if it exists).
The next command performs a Replace (without a replacement string) which will remove any carriage-returns from the previous field.
The Field Commands can also be used to read fixed length fields by using a number instead of a string.
FIELD TEST 130
Would populate the field TEST with the next 130 characters.
And the special EOF delimiter can be used to read everything until the end of the file, so
FIELD ALLDATA EOF
Would populate the field ALLDATA with the entire contents of the input file.
To produce the output from our example, we need to add the START-OUTPUT command and by using {fieldname} markers
each record would be printed substituting the markers with field values.
START-OUTPUT
Customer Number: {CUSTNUM}
Customer Name: {NAME}
Customer Location: {LOC}
This would be printed for each record, however, this would also include the first record of input which in our
example contains a header record for the input file. This can be exluded by adding the SKIPHEADER Command at the
top of the template.
Also, maybe we would like a heading on our output and a footer showing the total number of records.
This can be done as shown in the complete template example below (the markers and commands are discussed later in this document).
# Example template. (Lines beginning with # are comments)
SKIPHEADER
FIELD CUSTNUM ";"
FIELD NAME ";"
FIELD LOC "{:10}"
REPLACE "{:13}"
# Start Output
START-OUTPUT
Customer List
{{RECORDSTART}}
======================================
Customer Number: {CUSTNUM}
Customer Name: {NAME}
Customer Location: {LOC}
{{RECORDEND}}
The report shows {{RECORDNUM}} Customers
To run this report you would (from dos) run ...
fileconvert -I"mydata.csv" -T"sample.txt" -O"mydata.txt"
Which would try to read mydata.csv using the template sample.txt to produce the output file mydata.txt
which would contain the following ...
Customer List
======================================
Customer Number: 001
Customer Name: Joe Bloggs
Customer Location: London
======================================
Customer Number: 002
Customer Name: John Smith
Customer Location: Paris
The report shows 2 Customers
The following sections describe all the available commands. although the structure is very simple it is extremely
powerfull and you can produce just about any type of output although the example above was pure text, we could have generated
xml, html or even binary (by using {:dec} markers. We could also do more complex replacements on the input data and could
have applied filters to restrict the data output, used markers to define a multi-page report or processed multiple input
files in one go.
Template Commands
The START-OUTPUT Command must be defined in all templates. Everything above this command is referred to as the
input section (defines input data and what to do with it) and everything below this command is referred to as the
output section (defines the output to be produced).
Template Commands -Input Section
The following commands can be used within a template file and define how the input is read and processed.
Please also refer to the sections on special Characters and Passing Parameters as these markers can be used
throughout your template including this section.
#
Defines a comment line (ignored)
TRANSLATE-INPUT "fromchar" ["tochar"]
This command can only be used before any field definitions and allows you to perform a character conversion
at the time the input is read and before any processing occurs. If the tochar is missing then the fromchar
will be removed from the input stream.
Example:
TRANSLATE-INPUT "{:9]" " "
will Replace Tab Characters with a space during the reading of the input file.
DEFAULT-Pn
Defines a default value for the variable {P1} to {P5}.
The values here will be overwritten if the -Pn parameter is used to pass a value from the command line.
See the section passing parameters for more information on this command.
Example
DEFAULT-P1 "Our Customers"
SPECIAL val
When processing binary files, problems can occur if the data contains bytes containing special characters (Ascii Code < 9). This command allows you to remove these bytes from the input stream or replace
them with another code. (by default the program will replace these characters with a space.
Example:
SPECIAL REMOVE
SPECIAL 9
The first option will cause special Characters to be removed (slower than the second option), and the second will replace special characeters with a 9 (Tab).
SKIPHEADER [lines]
This command can be used if your input file contains a header row which you want to ignore in the outputs.
if you wish you can enter a number to ignore several lines
Example:
SKIPHEADER 3
will ignore the first 3 input records.
SET-EMBEDDED-STRINGS
This command (if used) must appear before any field definitions and can be usefull when reading some types
of file as possible delimiters embedded in strings will be ignored.
For example:
If you are reading a file using a semi colon as a field delimiter and the file contains ..
001;"data";" a description; comment"
You could have problems as the delimter after the description may cause the field to be extracted incorrectly.
Adding this command will result in the program detecting that this parameter is embedded in a field string "..." and
will ignore it during extraction and so will split the field correctly.
TRIM-MULTIPLE-SPACES
This command (if used) must appear before any field definitions and will replace multiple spaces with a single space
when reading the input file (this will happen after any translate commands have been applied).
MAKE-LOWERCASE
Will convert data to Lower Case.
If the command appears before any field definitions, then it will be applied to all input and output, otherwise it will only be applied to the previously defined field, but note that global replaces take precedence over field replacements
MAKE-UPPERCASE
Will convert data to Upper Case.
If the command appears before any field definitions, then it will be applied to all input and output, otherwise it will only be applied to the previously defined field, but note that global replaces take precedence over field replacements
EXCLUDE-FILTER
When using the FILTER Commands any data containing any of the filter strings will be included in the outputs.
This command allows you to reverse this processing so that only records that DO NOT contain any of the Filter Strings
will be output.
You only need to enter this command once and it will apply to ALL Filters
Example:
EXCLUDE-FILTER
FILTER "test"
Only records that DO NOT contain the string test will be printed.
FILTER-IGNORE-CASE
By Default, FILTER Commands are case sensitive. This command will set all filters to case-insesitive search mode.
You only need to enter this command once and it will apply to ALL Filters. Please not that this command can also be
passed from the command line using the -FC parameter.
Example:
FILTER-IGNORE-CASE
FILTER "test"
The filter will return records containing TEST and will ignore case when searching.
FILTER string [DISPLAY string]
If this command appears before any field defintions then only records containing this value will be output.
If the command appears below a field definition then only records with the field containing this value will be output.
You can have as many of these filter commands as you need, the record will be output if any condition returns true and the {{FILTER}}
marker can be used in the output section to display the filter condition met.
Also refer to FILTER-IGNORE-CASE and EXCLUDE-FILTERS Commands above.
By default the filter value will be displayed in the {{FILTER}} marker, however, you can use the optional DISPLAY option to display a
different value in this marker.
Example:
FILTER "test"
FILTER "test" DISPLAY "contains 'test'"
REPLACE Commands
If the replace commands appear before any field definitions, they will be applied to all input and output, otherwise they will only be applied to the previously defined field.
There are several replace commands available, each of which is described below
[IGNORE-CASE] REPLACE “string1”
Will remove all occurrances of string1. The optional IGNORE-CASE allows you to ignore the case when searching.
Examples:
REPLACE “hello”
If input contains “hello world HELLO you” then the result would be “world and HELLO you”
IGNORE-CASE REPLACE “hello”
If input contains “hello world HELLO you” then the result would be “world you”
[IGNORE-CASE] REPLACE “string1” WITH “string2”
Will replace all occurrances of string1 with string2. The optional IGNORE-CASE allows you to ignore the case when searching.
Examples:
REPLACE “hello” WITH “bye’
If input contains “hello world HELLO you” then the result would be “bye world and HELLO you”
IGNORE-CASE REPLACE “hello” WITH “bye”
If input contains “hello world HELLO you” then the result would be “bye world bye you”
REPLACE-ALL ...
Identical to REPLACE command described above
REPLACE-UPTO-LAST [IGNORE-CASE] REPLACE-UPTO-LAST "string1" [WITH “string2"]
Identical to the previously discussed replace command except that it will replace all characters upto the last occurrance of "string1" with "string2"
Example:
REPLACE-UPTO-LAST "\" WITH "*"
would change "c:\temp\myfile.dat" to "*myfile.dat"
REPLACE-UPTO-FIRST [IGNORE-CASE] REPLACE-UPTO-FIRST "string1" [WITH "string2"]
Identical to the previously discussed replace command except it Will replace all characters
upto the first occurrance of "string1" with "string2"
Example:
REPLACE-UPTO-FIRST "\" WITH "*"
would change "c:\temp\myfile.dat" to "*temp\myfile.dat"
REPLACE-AFTER-LAST [IGNORE-CASE] REPLACE-AFTER-LAST "string1" [WITH "string2"]
Identical to the previously discussed replace command except that it will replace all characters from the last occurrance of "string1" with "string2"
Example:
REPLACE-AFTER-LAST "\" WITH "*"
would change "c:\temp\myfile.dat" to "c:\temp*"
REPLACE-AFTER-FIRST [IGNORE-CASE] REPLACE-AFTER-FIRST "string1" [WITH "string2"]
Identical to the previously discussed replace command except that it Will replace all characters from the first occurrance of "string1" with "string2"
Example:
REPLACE-AFTER-FIRST "\" WITH "*"
would change "c:\temp\myfile.dat" to "c:*"
REPLACE-FIRST [IGNORE-CASE] REPLACE-FIRST "string1" [WITH "string2"]
Identical to the previously discussed replace command except that it will replace first occurrance of "string1" with "string2"
Example:
REPLACE-FIRST "\" WITH "*"
would change "c:\temp\myfile.dat" to "c:*temp\myfile.dat"
REPLACE-LAST [IGNORE-CASE] REPLACE-LAST "string1" [WITH "string2"]
Identical to the previously discussed replace command except that it will replace last occurrance of "string1" with "string2"
Example:
REPLACE-LAST "\" WITH "*"
would change "c:\temp\myfile.dat" to "c:\temp*myfile.dat"
FIELD Definitions
Field Definitions define exactly how the data is read from the input file. Several syntax options are available which are described
below.
.
The optional MANDATORY keyword allows to you make the field mandatory, in this case the record will not be output if this field is blank.
It is worth noting at this point that fileconvert does not have a command specific to an End of Record, instead, it always usese the last
field definition (except COPY-FIELD's) to define the end of record.
Note: You may also want to use the SET-EMBEDDED-STRINGS command if you are reading CSV Files.
[MANDATORY] FIELD FIELDNAME "string" [outlen]
Defines a delimited field. The optional outlen allows you to pad the field length to force specific output size.
The optional MANDATORY keyword allows to you make the field mandatory, in this case the record will not be output if this field is blank.
Note: You may also want to use the SET-EMBEDDED-STRINGS command if you are reading CSV Files.
Example:
FIELD CUST_NAME ";" 30
FIELD CUST_NUM ";"
FIELD LOC "{:13}{:10}"
The Field CUST_NAME will contain all text before the ; character.
It will be sized to 30 characters before being output.
The next field is CUST_NUM which is delimited by the next ;.
The last field will be populated with data upto the carriage-return+linefeed characters which is how a DOS file is delimited. For
more information on the {: Tags please refer to the Special Characters Section.
This field will not be resized.
[MANDATORY] FIELD FIELDNAME size [outlen]
This syntax can be used for fixed length input fields and allows you to specify the input data size of the field.
The optional MANDATORY keyword allows to you make the field mandatory, in this case the record will not be output if this field is blank.
Example:
FIELD CUST_LOC 30
The field CUST_LOC takes the data from the next 30 characters.
[MANDATORY] FIELD FIELDNAME EOF [outlen]
This syntax can be used to specify that the remainder of the input file should be used to populate this field.
The optional MANDATORY keyword allows to you make the field mandatory, in this case the record will not be output if this field is blank.
Example:
FIELD WHOLE_FILE EOF
The field WHOLE_FILE will contain the entire contents of the input file.
COPY-FIELD FIELDNAME
Defines field that is a copy of the previously defined FIELD.
This is usefull for making a copies of a field prior to performing replacements on the data.
Example:
FIELD CUST_NAME ";" 30
COPY-FIELD NEWNAME
The Field NEWNAME will be defined. Its value will be copied from the CUST_NAME Field before any replace commands
are executed.
START-OUTPUT [UNIQUE] [SORT[DESC]] [PAGERECS pagerecs] [maxrecs]
This defines the start of the output section.
The optional SORT or SORTDESC command allows you to sort your output
in ascending or descending order respectively.
Note that if you use this option the report will run in Buffered Output Mode and also note that this
option Cannot be used for multi-page reports.
Example:
START-OUTPUT SORTDESC
The optional UNIQUE command allows you to specify that only unique records will be output (duplicates will
be ignored).
Example:
START-OUTPUT SORT UNIQUE
START-OUTPUT UNIQUE
First option will sort the output and only output unique records.
The second option will not perform a sort, but will only output unique records.
The optional maxrecs value allows you to specify how many records will be processed.
Example:
START-OUTPUT
Will produce the report for all records
START-OUTPUT 10
Will only produce output for 10 Records.
You can also use this parameter to turn your report into a multi-page report.
The PAGERECS option allows you to specify the number of records to be displayed on each page of the
report.
the page number will be added to the end of the output filename you specified at runtime for each file generated.
Note that the -B (Buffered Output) command line option can not be used for multi-page reports.
Example:
START-OUTPUT PAGERECS 10
Will produce a new output file for each 10 records
START-OUTPUT PAGERECS 10 20
Same as above, except the report will stop after 20 records
Everything below the START-OUTPUT Marker is treated as an output definition.
Any text written here will be included in the outout and as such this section does not contain any commands.
instead, you embed markers into your text. Fileconvert will replace these markers with values when producing the
output
Please also refer to the sections on special Characters and Passing Parameters as these markers can be used
throughout your template.
To define a field value in your output simply embrace the fieldname with single wavy brackets.
example:
The value for Customer name is {CUST_NAM}.
The {CUST_NAM} will be replaced with the current value of the field CUST_NAM.
Additionally, the following special markers can also be used in the output section to tell fileconvert when the
data is to be displayed...
{{REPORTHEADEREND}}
Usually used in mutli-page reports.
everything before this marker will be put at the top of the first page of the report and everything below
it down to the {{RECORDSTART}} tag will be put on subsequent pages (ie: Not the first page).
If the tag is missing then everything above {{RECORDSTART}} will be printed on each page.
{{RECORDSTART}}
Everything before this will be included at the beginning of the output.
(also see comments on {{REPORTHEADEREND}} marker above).
{{RECORDEND}}
everything between {{RECORDSTART}} and this marker will be output for each record
Everything after this marker will be output at the end of the output (also see {{REPORTFOOTSTART}} marker below).
{{REPORTFOOTSTART}}
Usually used in mutli-page reports.
Everything between the {{RECORDEND}} and this marker will be printed on every page except the last one.
Everythnig below this marker will be printed once at the end of the report.
If the marker is missing then everything below {{RECORDEND}} will be printed at the end of each page.
Additional Markers
The following markers can also be used anywhere in the output section.
{{RECORDNUM}}
This will be replaced with the Record Number in the current page if placed in the footer section it can be used as a total records on page count
{{RECORDTOT}}
This will be replaced with the Record Number in the report (ie: Does not get cleared when new pages printed). if placed in the report footer section it can be used as a total records on report count
{{PAGENUM}}
This will be replaced with the number of the current page
{{OUTPUT-FILENAME}}
This will be replaced with the full filename of the current output file unless placed in the footer blocks in which case it will show the next filename (usefull for next page links in multi-page html reports)
{{OUTPUT-SHORTNAME}}
This works the same as the previous tag, except that it is replaced by the filename with the directory and path removed.
{{INPUT-FILENAME}}
This will be replaced with the full filename of the current input file.
{{INPUT-SHORTNAME}}
This will be replaced with the short filename of the current input file (ie: without the file path).
{{FILTER}}
This marker can ONLY be used within the Record Section (between {{RECORDSTART}} & {{RECORDEND}} markers) and will be replaced
with the value of the command line -F filter or FILTER Commands that was matched. If filters are not being used
or the EXCLUDE-FILTER Filter command is used when running the report then this marker will be removed automatically..
NOTE:When you use this marker the record will be printed ONCE FOR EACH filter matched. Without the marker, the record
will be printed only once.
{STARTPOS FIELDNAME}
Outputs the Start position of the Field in the input record
Example:
{STARTPOS CUST_NAM}.
{ENDPOS FIELDNAME}
Outputs the position of the last character of the field in the input record.
Example:
{ENDPOS CUST_NAM}.
{LENGTH FIELDNAME}
Outputs the length of the field in the input record.
Example:
{LENGTH CUST_NAM}.
Special Characters
You can use {:dec} markers ANYWHERE within your template to specify non-printable characters.
The dec should be replaced with the ascii code of the character required and note that the colon is needed.
Some common examples are shown below
Using Replace to change TAB characters to spaces.
REPLACE "{:9}" WITH " "
Remove Carriage-Returns
REPLACE "{:13}"
Replace linefeeds with Carriage Returns
REPLACE "{:10}" WITH "{:13}"
Passing Parameters
You can pass parameters directly from the command line into your template.
Please refer to tdocumentation on the following commands:
Command line -Pn options
use these options to pass the parameters
Template Command DEFAULT-Pn
Allows you to define a default value for a Pn Parameter.
To access one of these parameters withing your template simply use the {Pn} markets ANYWHERE (they can also be used within the
input secion) in your template.
REPLACE "test" WITH "{P1}"
will replace the word test with the value in the P1 parameter.
Our Example Application
If you look into the directory where you installed fileconvert, you will see a file called quickconvert.bat.
This is a DOS based program that uses fileconvert to perform some usefull functions. The templates used by this program
are in the templates directory. Please feel free to run the program and/or examine the code to get a better
idea of what you can do with fileconvert. It can also be used to generate basic templates to get you started.
NOTE:Before running the program for the first time you will need to edit the file (using a simple text editor
such as notepad) and change the command..
SET FCDIR=c:\fileconvert
to the directory in which you installed this application.
This quickconvert program presents you with a menu allowing to perform the following actions:
Convert File(S) from Unix to DOS Format
DOS files containing several lines are delimited with a carriage-return + linefeed, whereas unix files use the
linefeed character only. This option allows you change one or more files from unix format to dos format.
Convert File(S) from DOS to Unix Format
DOS files containing several lines are delimited with a carriage-return + linefeed, whereas unix files use the
linefeed character only. This option allows you change one or more files from dos format to unix format.
Perform a Search and Replace
Allows you to perform an search and replace across multiple files. also allows you to include special characters.
Perform a Search
Allows you to perform a search in multiple files. also allows you to include special characters.
Convert a CSV File and generate a Template
Allows you to sort CSV Files or convert them to html. This option also generates a template and so
is a good way of getting the hang of how fileconvert works.
Example Templates
This section contains a few complete templates to help you understand how fileconvert works.
Template to read a CSV File to produce HTML
This template reads the CSV File below and produces an html Page as output.
CUST_NUM;CUST_NAM;CUST_INVAMT;CUST_LOC
001;"John Smith";23.13;"London"
004;"Mickey Mouse";100.00;"Paris"
002;"Joe Bloggs";5;"New York"
005;"Joe Smaile";54E;"Madrid"
Template
# Template to read sample1.dat data file and produce html page output
# =========================================================================
# Demonstrates reading a CSV File.
# Also Demonstrates the use of Aggregate functions and the P1 parameter
# Default Value for P1 parameter if -P1 not passed on the command line.
DEFAULT-P1 "Our Customers"
#
# SKIPHEADER tells program that first record is a header record which will not be output.
# You could enter SKIPHEADER 2 to skip the first 2 lines.
SKIPHEADER
MAKE-LOWERCASE
# The Mandatory Keyword tells the program to ignore this record is the CUST_NUM Field is blank
# The first 2 Fields are delimited with a semi-colon and the last field uses the record delimiter (carriage return + linefeed)
# The Last (optional) parameter allows you to force the output width of the field. In the case of CUST_LOC I have set this
# to 3 which will result in only the first 3 characters being displayed. (if it is a large number, eg: 10, then the output
# will be padded with spaces to this size).
# Field Name Delimiter Output Size
# ========== ========= ===========
MANDATORY FIELD CUST_NUM ";"
FIELD CUST_NAM ";"
# Remove " from CUST_NAM Field
REPLACE '"'
FIELD CUST_INVAMT ";"
FIELD CUST_LOC "{:13}{:10}" 3
# Remove " from CUST_LOC Field
REPLACE '"'
# Everything below the START-OUTPUT defines the report output.
# The area before {{RECORDSTART}} will be printed once at the start of the report
# The area after {{RECORDEND}} will be printed once at the end of the report
# The area between these 2 markers will be printed once for each record.
#
# The {fieldname} markers will have values subsituted with data values.
# The {RECORDNUM} marker will be replaced with the record count.
START-OUTPUT
<html>
<title>Fileconvert - Sample 1 Report</title>
<body>
<h2>Sample Report - {P1}</h2>
<table>
<tr>
<td><em>Customer Number</em></td>
<td><em>Customer Name</em></td>
<td><em>Invoice Amount</em></td>
<td><em>City</em></td>
</tr>
<tr>
<td><hr></td>
<td><hr></td>
<td><hr></td>
<td><hr></td>
</tr>
{{RECORDSTART}}
<tr>
<td><strong>{CUST_NUM}
</strong></td>
<td><em>{CUST_NAM}</em></td>
<td><em>{CUST_INVAMT}</em></td>
<td><em>{CUST_LOC}</em></td>
</tr>
{{RECORDEND}}
<tr>
<td><hr></td>
<td><hr></td>
<td><hr></td>
<td><hr></td>
</tr>
</table>
<br/>
Total Clients: <strong>{{RECORDNUM}}</strong>
Template to read a CSV File to produce a Multi Page HTML Output
This template reads the CSV File below and produces a multi page html as output.
CUST_NUM;CUST_NAM;CUST_INVAMT;CUST_LOC
001;"John Smith";23.13;"London"
004;"Mickey Mouse";100.00;"Paris"
002;"Joe Bloggs";5;"New York"
005;"Joe Smaile";54E;"Madrid"
It is similier to the previous example, except for the output section which contains the
PAGERECS option to tell fileconvert how many records to display on each page and some additional
markers (Report Header, Page Header etc).
# Template to read sample1.dat data file and produce html multi-page output
# =========================================================================
# Demonstrates reading a CSV File.
# Almost the same as previous example but adapted to work as a multi-page report.
# Default Value for P1 parameter if -P1 not passed on the command line.
DEFAULT-P1 "Our Customers"
#
# SKIPHEADER tells program that first record is a header record which will not be output.
# You could enter SKIPHEADER 2 to skip the first 2 lines.
SKIPHEADER
MAKE-LOWERCASE
# The Mandatory Keyword tells the program to ignore this record is the CUST_NUM Field is blank
# The first 2 Fields are delimited with a semi-colon and the last field uses the record delimiter (carriage return + linefeed)
# The Last (optional) parameter allows you to force the output width of the field. In the case of CUST_LOC I have set this
# to 3 which will result in only the first 3 characters being displayed. (if it is a large number, eg: 10, then the output
# will be padded with spaces to this size).
# Field Name Delimiter Output Size
# ========== ========= ===========
MANDATORY FIELD CUST_NUM ";"
FIELD CUST_NAM ";"
# Remove " from CUST_NAM Field
REPLACE '"'
FIELD CUST_INVAMT ";"
FIELD CUST_LOC "{:13}{:10}" 3
# Remove " from CUST_LOC Field
REPLACE '"'
# Everything below the START-OUTPUT defines the report output.
# The area before {{RECORDSTART}} will be printed once at the start of the report
# The area after {{RECORDEND}} will be printed once at the end of the report
# The area between these 2 markers will be printed once for each record.
#
# The {fieldname} markers will have values subsituted with data values.
# The {RECORDNUM} marker will be replaced with the record count.
START-OUTPUT UNIQUE PAGERECS 2
<html>
<title>Fileconvert - Sample 1 Report</title>
<body>
<h2>Sample Report - {P1} Page {{PAGENUM}}</h2>
<table>
<tr>
<td><em>Customer Number</em></td>
<td><em>Customer Name</em></td>
<td><em>Invoice Amount</em></td>
<td><em>City</em></td>
</tr>
<tr>
<td><hr></td>
<td><hr></td>
<td><hr></td>
<td><hr></td>
</tr>
{{RECORDSTART}}
<tr>
<td><strong>{CUST_NUM}</strong></td>
<td><em>{CUST_NAM}</em></td>
<td><em>{CUST_INVAMT}</em></td>
<td><em>{CUST_LOC}</em></td>
</tr>
{{RECORDEND}}
<tr>
<td><hr></td>
<td><hr></td>
<td><hr></td>
<td><hr></td>
</tr>
</table>
<a href="{{OUTPUT-SHORTNAME}}">Click here to view next page</a>
</body>
</html>
{{REPORTFOOTSTART}}
<tr>
<td><hr></td>
<td><hr></td>
<td><hr></td>
<td><hr></td>
</tr>
</table>
</body>
</html>
Template to read a fixed length File to produce HTML
This template reads the fixed length File below and produces an html Page as output.
001John Smith London,England
002Joe Bloggs London,England
004Mickey Mouse Paris,France
Template
# sample2.txt
#
# Template to read sample2.dat data file and produce html page output
# ===================================================================
# Demonstrates reading a Fixed Record Length File.
# Also demonstrates the use of the COPY-FIELD Command
# The Mandatory Keyword tells the program to ignore this record is the CUST_NUM Field is blank to skip blank records.
# As a number is presented as the Delimiter the program will treat the input as fixed length fields.
#
#
# Field Name Delimiter Output Size
# ========== ========= ===========
MANDATORY FIELD CUST_NUM 3
FIELD CUST_NAM 13
FIELD CUST_LOC 14
# As the Location contains "City,Country" we can copy the field to CITY and COUNTRY
# and then use the replace commands to remove the unwanted part.
COPY-FIELD CITY
REPLACE-AFTER-FIRST ","
COPY-FIELD COUNTRY
REPLACE-UPTO-FIRST ","
FIELD DUMMY 2
# Everything below the START-OUTPUT defines the report output.
# The area before {{RECORDSTART}} will be printed once at the start of the report
# The area after {{RECORDEND}} will be printed once at the end of the report
# The area between these 2 markers will be printed once for each record.
#
# The {fieldname} markers will have values subsituted with data values.
# The {RECORDNUM} marker will be replaced with the record count.
START-OUTPUT
<html>
<title>Fileconvert - Sample 2 Report</title>
<body>
<h2>Sample Report</h2>
<table>
<tr>
<td><em>Customer Number</em></td>
<td><em>Customer Name</em></td>
<td><em>City</em></td>
<td><em>Country</em></td>
</tr>
<tr>
<td><hr></td>
<td><hr></td>
<td><hr></td>
<td><hr></td>
</tr>
{{RECORDSTART}}
<tr>
<td><strong>{CUST_NUM}</strong></td>
<td><em>{CUST_NAM}</em></td>
<td><em>{CITY}</em></td>
<td><em>{COUNTRY}</em></td>
</tr>
{{RECORDEND}}
<tr>
<td><hr></td>
<td><hr></td>
<td><hr></td>
<td><hr></td>
</tr>
</table>
<br/>
Total Clients: <strong>{{RECORDNUM}}</strong>
</body>
</html>