Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

This page covers how to define the File Format of a text document to be used as a source or target. 

Creating a New File Format

Typically, File Formats are generated as part of the Wizard walkthrough when creating your integration. You can also generate a new File Format separately via one of the following methods:

  • Go to File > New > New File Format.
  • Click the  icon in the toolbar.
  • Right-click on File Formats in the left pane of Studio, then choose New File Format from the menu.

Your new File Format will be listed under the File Formats section in the left pane of Studio. In addition, a screen to configure your new File Format will open in the right pane of Studio. Here you choose whether to specify the structure manually or from a sample file:

  • Create Manually: Manually type in the names and types of the fields while configuring your File Format.

For File Formats being created manually, you will also need to select the document type, as well as the format.

  • Simple Text Document: Simple files have the same data row type throughout the text document. 
  • Complex Text Document: Complex files contain multiple file row types such as header or summary rows.

Both simple and complex document types allow character delimited and fixed field width formats.

Click the  button to continue to the next configuration screen.

You can also get back to the existing configuration at any time from the left pane of Studio under File Formats.

Configuring Fields Manually

After choosing to create your File Format manually, another configuration screen will be used to define each field in your document. This next step depends on the type of text structure you selected – simple or complex – as well as the format you selected – character-delimited or fixed field width.

Simple Text Document – Character-Delimited

If you have selected a simple text document that is character-delimited, your configured screen should look similar to that below. Following is an explanation for each configurable section.

Delimiter & Qualifier

  • Field Delimiter: Enter the character(s) to use as field delimiter(s) that specify the boundary between data fields. The default is a comma. You can also use multiple-character delimiters, such as a double pipe ||. To use a tab as a delimiter, enter /\t.
  • String Qualifier & Use qualifier: Enter a character (the default is a double quote) in the String Qualifier field. This field is a wrapper (usually double quotes) to indicate a string of words to be treated as a data unit. Also select an option next to the corresponding Use qualifier business rule: "Always" or "Only when needed." The default "Only when needed" is recommended.
  • Use line breaks for end of records: As of version 8.19, this checkbox is used to toggle on or off the use of line breaks as a separation between records. By default, end of records are denoted by line breaks so that each record exists on one line (unless using a qualifier). When disabled, line breaks are ignored and new records begin after the total number of fields are reached.

    Info
    NOTE: This flag is relevant only for source file formats. For target files, line breaks are always used to denote end of records.
  • Convert escape sequences: As of version 8.23, this checkbox is used to convert or ignore escape sequences, as indicated by a backslash (\). By default, escape sequences are converted in accordance with Java standards. When disabled, text conversion occurs with no interpretation of escape sequences.

    Info
    NOTE: This flag is relevant only for source file formats.

Validation (Optional)

  • Row must contain at least 'X' columns: If you want to use text validation (optional), you can enter a number of columns here that each row must contain, in order for the action specified next to occur.

    Note
    NOTE: You must have at least one row of data defined first in order to enable the validation options.
  • If not: After entering a number of columns, a dropdown will become available, where you can select to "Skip Row" or "Raise Error." More information on validation is available at Flat File Field Validation.

Define Segment Properties

Click the New button to define the document's first field. You must enter the following (required):

  • Field Name: Enter a name for this field.
  • Type: Select a type from the dropdown menu. Options include String, Integer, Date, Long, Float, Double, Boolean.

    Tip
    NOTE: If in doubt about the field type, it is recommended to choose String, since this type allows any data type to be included as a string.

The following are optional:

  • Default: If you want this field to have a default value, enter the value here.
  • Format: If you want this field to have a specific format, such as for date or currency, enter the format here.
  • Validation: Double-click in this column or use the  button on the far right to open additional options for field validation. More information on validation is available at Flat File Field Validation.

Repeat the previous steps for each additional data field you need to define.

Simple Text Document – Fixed Field Widths

If you have selected a simple text document that has fixed field widths, your configured screen should look similar to that below. Following is an explanation for each configurable section.

Validation (Optional)

  • Row must contain at least 'X' columns: If you want to use text validation (optional), you can enter a number of columns here that each row must contain, in order for the action specified next to occur.

    Note
    NOTE: You must have at least one row of data defined first in order to enable the validation options.
  • If not: After entering a number of columns, a dropdown will become available, where you can select to "Skip Row" or "Raise Error." More information on validation is available at Flat File Field Validation.

Define Segment Properties

Click the New button to define the document's first field. You must enter the following (required):

  • Field Name: Enter a name for this field.
  • Type: Select a type from the dropdown menu. Options include String, Integer, Date, Long, Float, Double, Boolean.

    Tip
    NOTE: If in doubt about the field type, it is recommended to choose String, since this type allows any data type to be included as a string.
  • Begin: Enter the starting character position (1 is considered to be the position for the beginning of the file). Note that the default value for subsequent rows will always be the next character after the previous field's ending position.

  • Length: Enter the number of characters that are reserved for this field.

    Tip
    NOTE: Fixed-length text files always specify a certain number of characters that are reserved for each field. For example, you may identify a field called LastName that always has 30 characters reserved for it. (If the actual value is Smith, only 5 characters are used; the remaining characters have nothing in them.)

    If you have changed any field's character length value, use the  button to recalculate all the fields' starting positions.

    Warning

    WARNING: The Calculate button resets all the Begin and End field values, based on the Length values you specified. Do not use this button if your file's field positions have gaps or overlaps, as they will be deleted.

  • End: Enter the ending character position.

    Info
    NOTE: By default, fixed-length segments are terminated with a newline character.

The following are optional:

  • Default: If you want this field to have a default value, enter the value here.
  • Format: If you want this field to have a specific format, such as for date or currency, enter the format here.
  • Validation: Double-click in this column or use the  button on the far right to open additional options for field validation. More information on validation is available at Flat File Field Validation.

Repeat the previous steps for each additional data field you need to define.

Complex Text Document – Character-Delimited

If you have selected a complex text document that is character-delimited, your configured screen should look similar to that below. Following is an explanation for each configurable section.

Delimiter & Qualifier

  • Field Delimiter: Enter the character(s) to use as field delimiter(s) that specify the boundary between data fields. The default is a comma. You can also use multiple-character delimiters, such as a double pipe ||. To use a tab as a delimiter, enter /t.

  • String Qualifier & Use qualifier: Enter a character (the default is a double quote) in the String Qualifier field. This field is a wrapper (usually double quotes) to indicate a string of words to be treated as a data unit. Also select an option next to the corresponding Use qualifier business rule: "Always" or "Only when needed." The default "Only when needed" is recommended.

  • EDIFACT: Check this box if you are using EDIFACT syntax rules within your file. If you do not know what this option is, leave unchecked.

  • Use line breaks for end of records: As of version 8.19, this checkbox is used to toggle on or off the use of line breaks as a separation between records. By default, end of records are denoted by line breaks so that each record exists on one line (unless using a qualifier). When disabled, line breaks are ignored and new records begin after the total number of fields are reached.

    Info
    NOTE: This flag is relevant only for source file formats. For target files, line breaks are always used to denote end of records.
  • Convert escape sequences: As of version 8.23, this checkbox is used to convert or ignore escape sequences, as indicated by a backslash (\). By default, escape sequences are converted in accordance with Java standards. When disabled, text conversion occurs with no interpretation of escape sequences.

    Info
    NOTE: This flag is relevant only for source file formats.

Define Segments

Click the New button to define the document's first segment. You must enter the following (required):

  • Segment Name: Enter a name for the segment.
  • Segment Parent: Identify the parent of this segment from the dropdown menu.
  • Occurrence: Specify how the segment will occur in your complex document by choosing one of the options from the dropdown menu:
    • Choose Only Once if this segment must occur once but only once in the document.
    • Choose One or More if this segment must occur at least once, but can also be repeated.
    • Choose Zero or One if this segment may not occur, but if it does, it can only occur once.
    • Choose Zero or More if this segment may not occur, but if it does, it can occur one or more times. 

Repeat the previous steps for each additional segment you need to define.

Define Segment Properties

Now that all the segments have been created, you will need to define each segment's fields.

Either double click on the desired segment's # column, or with the segment highlighted click the  button. The Segment: Detail popup will appear.


For each segment, click the New button to define the segment's fields. You must enter the following (required):

  • Field Name: Enter a name for this field.
  • Type: Select a type from the dropdown menu. Options include String, Integer, Date, Long, Float, Double, Boolean.

    Tip
    NOTE: If in doubt about the field type, it is recommended to choose String, since this type allows any data type to be included as a string.

The following are optional:

  • Default: If you want this field to have a default value, enter the value here.
  • Format: If you want this field to have a specific format, such as for date or currency, enter the format here.
Note
NOTE: Each segment must include one and only one Identifier field. Make sure that you have one field whose data type is set to this.

Repeat the previous steps for each additional data field you need to define.

Complex Text Document – Fixed Field Widths

If you have selected a complex text document that has fixed field widths, your configured screen should look similar to that below. Following is an explanation for each configurable section.

Define Segments

Click the New button to define the document's first segment. You must enter the following (required):

  • Segment Name: Enter a name for the segment.
  • Segment Parent: Identify the parent of this segment from the dropdown menu.
  • Occurrence: Specify how the segment will occur in your complex document by choosing one of the options from the dropdown menu:
    • Choose Only Once if this segment must occur once but only once in the document.
    • Choose One or More if this segment must occur at least once, but can also be repeated.
    • Choose Zero or One if this segment may not occur, but if it does, it can only occur once.
    • Choose Zero or More if this segment may not occur, but if it does, it can occur one or more times. 

Repeat the previous steps for each additional segment you need to define.

Define Segment Properties

Now that all the segments have been created, you will need to define each segment's fields.

Either double click on the desired segment's # column, or with the segment highlighted click the  button. The Segment: Detail popup will appear.

For each segment, click the New button to define the segment's fields. You must enter the following (required):

  • Field Name: Enter a name for this field.
  • Type: Select a type from the dropdown menu. Options include String, Integer, Date, Long, Float, Double, Boolean.

    Tip
    NOTE: If in doubt about the field type, it is recommended to choose String, since this type allows any data type to be included as a string.
  • Begin: Enter the starting character position (1 is considered to be the position for the beginning of the file). Note that the default value for subsequent rows will always be the next character after the previous field's ending position.

  • Length: Enter the number of characters that are reserved for this field.

    Tip
    NOTE: Fixed-length text files always specify a certain number of characters that are reserved for each field. For example, you may identify a field called LastName that always has 30 characters reserved for it. (If the actual value is Smith, only 5 characters are used; the remaining characters have nothing in them.)

    If you have changed any field's character length value, use the  button to recalculate all the fields' starting positions. 

    Warning

    WARNING: The Calculate button resets all the Begin and End field values, based on the Length values you specified. Do not use this button if your file's field positions have gaps or overlaps, as they will be deleted.

  • End: Enter the ending character position.

    Info
    NOTE: By default, fixed-length segments are terminated with a newline character.

The following are optional:

  • Default: If you want this field to have a default value, enter the value here.
  • Format: If you want this field to have a specific format, such as for date or currency, enter the format here.
Note
NOTE: Each segment must include one and only one Identifier field. Make sure that you have one field whose data type is set to this.

Repeat the previous steps for each additional data field you need to define.

Configuring Fields from Sample File

To create fields from a sample file, rather than manually, please refer to Creating a Text Document Definition from a Sample File.

Panel
borderColor#65379B
titleColor#FFFFFF
titleBGColor#65379B
titleOn This Page
Table of Content Zone

Table of Contents
maxLevel3
minLevel2
indent20px

Panel
borderColor#00B886
titleColor#FFFFFF
titleBGColor#00B886
titleRelated Topics
Content by Label
showLabelsfalse
max1000
showSpacefalse
sorttitle
cqllabel in ("source-overview","file-format-overview","target-overview") and space = "DOC"

HideElements
metastrue

Last updated: 

Lastmodifieddate