[Magento] How export and import are implemented?

Introduction

This long anticipated feature is arguably one of themost importantones
in ecommerce world. Flexibility of this interface definescapability of the application to communicate with the outsideworld, be it legacy systems, 3rd party applications, priceaggregators, external order sources, fulfillment systems,accounting, etc.

Thus we took it very seriously and, as with everything elsein Magento wentone
or more steps further than most systems. Effectivelyany import orexport
is a conversion.From one medium to another. From database to local file,from FTP serverto
database, from parsed HTML pageto
Excel spreadsheet, from Google Spreadsheet to catalog.

Basically, it boils down to loading data from one resource,applying unlimited alterations/reformatting, and saving it toanother resource.

Welcome to the world of conversion.

Concept

Let’s take an example of exporting a product catalog to excelspreadsheet file. The logical steps are:

Load records from database

Convert internal identificators into human understandabletext

Map database fields to our custom file headings (sku ⇒ PartNumber)

Format data array as Excel XML

Save resulting text as local or remote file

This sequence of actions we call a Profile.

Here we have 4 categories of components:

Adapter (input/output)

Mapper (data value processing)

Parser (data format conversion)

Validator (data validation)

Adapters

Adapters are anything that can plug in to external resource. Thismay include:

Local files

FTP,
SFTP servers

Database table

Soap services

Standard input/output

HTTP interface

HTML source

Google Spreadsheets

Custom resources (specialized db interface, cache repository,etc.)

Adapters can have 2 methods:

load

- extract datafrom resource into internal container.

save

- save data intoresource.

Mappers

Mappers are for processing data without changing it’s format orstructure. They’re useful for tasks such as mapping one field toanother or apply formulas on values.

Mappers can run one method:

map

Parsers

Parsers are for changing format of data. Examples:

CSV text ⇒ Grid array

Excel XML text⇒
Grid array

Grid array ⇒ Product model collection

Parsers can have 2 methods:

parse

- usually convertfrom more human readable data to machine
friendly format

unparse

- the opposite,into human readable format

You might notice that Parsers are separated from Adapters, whenusually they are associated as one thing (export to CSV file).

Validators

Validators are created to make sure that conditions are met. Theconditions can be applied to processed data or environmentvariables.

Validators can only

validate

.

Implementation

Let’s take an example of exporting catalog products into CSVfile.

<action type="catalog/convert_adapter_product" method="load">

<var name="store">base</var>

</action>

<action type="catalog/convert_parser_product" method="unparse">

</action>

<action type="varien/convert_mapper_column" method="map">

<var name="sku">Part#</var>

</action>

<action type="core/convert_parser_csv" method="unparse">

<var name="delimiter">,</var>

<var name="enclose"><![CDATA["]]></var>

<var name="fieldnames">true</var>

</action>

<action type="core/convert_adapter_io" method="save">

<var name="type">file</var>

<var name="path"></var>

<var name="filename">products.csv</var>

<var name="link">/dev/moshe/magento/var/export/products.csv</var>

</action>