NDJSON import

There exists a generic data exchange format to connect any system with Makaira. The data format is based on the NDJSON format. The format holds one record in one line which makes parsing easy and effective.

{"id": "1", "type": "product", "more": "values and so on"}
{"id": "2", "type": "variant", "more": "values and so on"}
{"id": "3", "type": "manufacturer", "more": "values and so on"}

📘
Note
For better clarity, the documents are displayed in multiple lines in the following documentation. In the final data feed each document must be delivered on separate lines without line breaks in the document itself.

General types

General types are use in every document type

Type	Description
timestamp	Timestamp at the time of export from the store (format: YYYY-MM-DD HH:MM:ss)
groups	A set of data that is overridden based on the constraint "group" in the API - e.g. useful for discount groups. But also for target group-based targeting. `"groups": {"DISCOUNT80": {"price": 2.00, "price_no_vat": 1.68}, "DISCOUNT800": {"price": 20.00, "price_no_vat": 1.68}}`

Type

Description

timestamp

Timestamp at the time of export from the store
(format: YYYY-MM-DD HH:MM:ss)

groups

A set of data that is overridden based on the constraint "group" in the API - e.g. useful for discount groups. But also for target group-based targeting.

"groups": {"DISCOUNT80": {"price": 2.00, "price_no_vat": 1.68}, "DISCOUNT800": {"price": 20.00, "price_no_vat": 1.68}}

Type product

{
  "id": "10013859950",
  "type": "product",
  "parent": "",
  "shop": "4",
  "ean": "48690A",
  "isVariant": false,
  "active": true,
  "stock": 0,
  "onstock": true,
  "picture_url_main": "https://images.makaira.io/out/pictures/generated/250_250/pboxx-pixelboxx-2652106/GRAPH%27IT+%22Brush+%26+Extra+fine%22+Marker+Komplett-Set%2C+Display+mit+96+Markern.jpg",
  "title": "GRAPH'IT \"Brush & Extra fine\" Marker Komplett-Set, Display mit 96 Markern",
  "shortdesc": "Das GRAPH'IT \"Brush & Extra fine\" enth\u00e4lt 96 Marker mit Qualit\u00e4tstinte auf Alkoholbasis und zwei unterschiedlichen Spitzen. Ideal geeignet f\u00fcr unterschiedlichste Mal- und Zeichentechniken.",
  "longdesc": "Das Komplett-Set enthält alle 96 Farbtöne der GRAPH'IT \"Brush & Extra fine\" Marker. Das transparente Display ermöglicht Ihnen jederzeit gute Übersicht und einen schnellen Zugriff zu Ihren Markern.Der GRAPH'IT \"Brush & Extra fine\" Marker verfügt über zwei unterschiedliche Spitzen. Eine lange, flexible Spitze mit synthetischen Pinselfasern, die eine optimale Strömung der Tinte gewährt. Und eine extrafeine 0,5 mm-Spitze mit Metallfassung, die sich insbesondere für Illustrationen eignet.GRAPH'IT \"Brush & Extra fine\" Marker \u2013 Die Produkt-Eigenschaften im ÜberblickKorpus aus weißem Kunststoff mit Dreikantform für hervorragende Haptik und Mal-KomfortQualitätstinte auf AlkoholbasisLeicht zu öffnender Deckel mit Farbcode zur einfachen IdentifizierungDer GRAPH'IT Marker ist ideal geeignet für:ComicsArchitekturDesignLandschaftsmalereiIllustrationuvm.",
  "price": 220.50,
  "soldamount": 0,
  "searchable": true,
  "searchkeys": "GRAPH'IT \"Brush & Extra fine\", Twin Tip, Marker, Pinselmarker, Pinselstift, Stift, Doppelmarker, Kalligrafie, Twin Marker, 2 Spitzen Maker, Comic, Architektur, Design, Illustration, GRAPH'IT Marker, Set, Komplett-Set, Brushpen, Brush Pen, Sortiment",
  "meta_keywords": "GRAPH'IT \"Brush & Extra fine\", Twin Tip, Marker, Pinselmarker, Pinselstift, Stift, Doppelmarker, Kalligrafie, Twin Marker, 2 Spitzen Maker, Comic, Architektur, Design, Illustration, GRAPH'IT Marker, Set, Komplett-Set, Brushpen, Brush Pen, Sortiment",
  "meta_description": "Das GRAPH'IT \"Brush & Extra fine\" Marker Komplett-Set finden Sie in unserem Onlineshop. \u2713 Marker mit Qualit\u00e4tstinte auf Alkoholbasis und zwei unterschiedlichen Spitzen. \u2713 Ideal f\u00fcr unterschiedlichste Mal- und Zeichentechniken. \u2713 \u00dcberzeugen Sie sich selbst! \u2794 Jetzt g\u00fcnstig online kaufen.",
  "manufacturerid": "f58f91bac483a6ea628e9d2be7a4b1e1",
  "manufacturer_title": "pebeo",
  "url": "/GRAPH-IT-Brush-Extra-fine-Marker-Komplett-Set-Display-mit-96-Markern.html",
  "maincategory": "9fb3ea945468f7c61b471c7401e7bad4",
  "maincategoryurl": "/Zeichnen-Grafik-Design/",
  "groups": {
    "DISCOUNT80": {
      "price": 2.00,
      "price_no_vat": 1.68
    },
    "DISCOUNT800": {
      "price": 20.00,
      "price_no_vat": 1.68
    }
  },
  "category": [
    {
      "catid": "cb38ac47b6005b97cd1bbed61c70fb18",
      "shopid": [
        "4"
      ],
      "path": "shop\/unser-angebot\/grafik-design\/stifte-marker\/copic-layoutmarker\/",
      "title": "Layoutmarker",
      "pos": 9 # position of product in category
    }
  ],
  "attributes": [
    {
      "d6238d67c4448e37c6bff847b097826e": [
        "Nur Neuheiten anzeigen"
      ],
      "5f65a4212ad959feb266af566d8f3b8f": [
        "Grafik + Design"
      ],
      "852264d68055531b4018323a7d2ba706": [
        "Einsteiger",
        "Schulen",
        "Studenten",
        "Hobbyk\u00fcnstler"
      ],
      "a33c9d88a89257f81eef0f4a001cc365": [
        "Marker + Fineliner"
      ],
      "4c9369a9a485d7ce007f58fa695e5812": [
        "Tinte auf Alkoholbasis"
      ],
      "d8008f87168439ac398f6927cdbd812d": [
        "Set"
      ],
      "6d9b20c89a0ffb28b56bb07ff5bb39ba": [
        "Pinselspitze",
        "metallgefasste Spitze"
      ],
      "76c7e398b67f441fdc9220e11b419945": [
        "0,5 mm"
      ]
    }
  ],
  "attributeStr": [
    {
      "id": "d6238d67c4448e37c6bff847b097826e",
      "title": "Neuheiten",
      "value": "Nur Neuheiten anzeigen"
    },
    {
      "id": "5f65a4212ad959feb266af566d8f3b8f",
      "title": "Empfohlen f\u00fcr Technik",
      "value": ["Grafik + Design", "Einsteiger", "Schulen", "Studenten", "Hobbyk\u00fcnstler"]
    },
    {
      "id": "a33c9d88a89257f81eef0f4a001cc365",
      "title": "Produktart",
      "value": "Marker + Fineliner"
    },
    {
      "id": "4c9369a9a485d7ce007f58fa695e5812",
      "title": "Marker-Basis",
      "value": "Tinte auf Alkoholbasis"
    },
    {
      "id": "d8008f87168439ac398f6927cdbd812d",
      "title": "Auswahl",
      "value": "Set"
    },
    {
      "id": "6d9b20c89a0ffb28b56bb07ff5bb39ba",
      "title": "Form Marker (Spitze)",
      "value": ["Pinselspitze", "metallgefasste Spitze"]
    },
    {
      "id": "76c7e398b67f441fdc9220e11b419945",
      "title": "Strichbreite",
      "value": "0,5 mm"
    }
  ],
  "attributeInt": [],
  "attributeFloat": [],
  "mak_boost_norm_insert": 0.07301991452214238,
  "mak_boost_norm_sold": 0.2763039747051342,
  "mak_boost_norm_rating": 0,
  "mak_boost_norm_revenue": 0.2664841710697083,
  "mak_boost_norm_profit_margin": 0,
  "timestamp": "2019-09-27 15:34:42",
}

Field definition

Field	Description
parent	This value must be empty for the product data type
attributeStr[], attributeInt[], attributeFloat[]	All attributes of the product Unsorted list of attributes, one entry per value (or values separated with unique delimiter - Makaira separates them during import) depending on the value type into the respective array: String (attributeStr) / Integer (attributeInt) / Float (attributeFloat) for more information see section Attribute Handling
attributes[]	Contains all attributes of the variants, grouped by variant (in the example only one variant exists) id: Attribute identifier value: List of attribute values (or values separated with unique delimiter - we separate them during import) List of attribute values (or values separated with unique delimiter - we separate them during import) for more information see section Attribute Handling
mak_boost_norm	Fields with logarithmically normalized values for: Novelty* (mak_boost_norm_insert) Number of sales* (mak_boost_norm_sold) Rating* (mak_boost_norm_rating) Sales* (mak_boost_norm_revenue) Margin* (mak_boost_norm_profit_margin)

Type variant

❗️
At least one variant is always required
Every product must have at least one child of type variant. Products without variants are not indexed correctly and will be missing from search results. If your source system has no variant concept, see Products without variants (pseudo variants) below.

{
  "id": "10013860005",
  "type": "variant",
  "parent": "10013859950",
  "shop": "4",
  "ean": "48690",
  "isVariant": true,
  "active": true,
  "sort": 4001,
  "stock": 10,
  "onstock": true,
  "picture_url_main": "https://images.makaira.io/out/pictures/generated/250_250/pboxx-pixelboxx-2616996/GRAPH%27IT+%22Brush+%26+Extra+fine%22+Marker+Komplett-Set%2C+Display+mit+96+Markern.jpg",
  "title": "GRAPH'IT \"Brush & Extra fine\" Marker Komplett-Set, Display mit 96 Markern",
  "shortdesc": "Das GRAPH'IT \"Brush & Extra fine\" enth\u00e4lt 96 Marker mit Qualit\u00e4tstinte auf Alkoholbasis und zwei unterschiedlichen Spitzen. Ideal geeignet f\u00fcr unterschiedlichste Mal- und Zeichentechniken.",
  "longdesc": "Das Komplett-Set enthält alle 96 Farbtöne der GRAPH'IT \"Brush & Extra fine\" Marker. Das transparente Display ermöglicht Ihnen jederzeit gute Übersicht und einen schnellen Zugriff zu Ihren Markern.Der GRAPH'IT \"Brush & Extra fine\" Marker verfügt über zwei unterschiedliche Spitzen. Eine lange, flexible Spitze mit synthetischen Pinselfasern, die eine optimale Strömung der Tinte gewährt. Und eine extrafeine 0,5 mm-Spitze mit Metallfassung, die sich insbesondere für Illustrationen eignet.GRAPH'IT \"Brush & Extra fine\" Marker \u2013 Die Produkt-Eigenschaften im ÜberblickKorpus aus weißem Kunststoff mit Dreikantform für hervorragende Haptik und Mal-KomfortQualitätstinte auf AlkoholbasisLeicht zu öffnender Deckel mit Farbcode zur einfachen IdentifizierungDer GRAPH'IT Marker ist ideal geeignet für:ComicsArchitekturDesignLandschaftsmalereiIllustrationuvm.",
  "price": 220,
  "soldamount": 0,
  "searchable": true,
  "searchkeys": "GRAPH'IT \"Brush & Extra fine\", Twin Tip, Marker, Pinselmarker, Pinselstift, Stift, Doppelmarker, Kalligrafie, Twin Marker, 2 Spitzen Maker, Comic, Architektur, Design, Illustration, GRAPH'IT Marker, Set, Komplett-Set, Brushpen, Brush Pen, Sortiment",
  "meta_keywords": "GRAPH'IT \"Brush & Extra fine\", Twin Tip, Marker, Pinselmarker, Pinselstift, Stift, Doppelmarker, Kalligrafie, Twin Marker, 2 Spitzen Maker, Comic, Architektur, Design, Illustration, GRAPH'IT Marker, Set, Komplett-Set, Brushpen, Brush Pen, Sortiment",
  "meta_description": "Das GRAPH'IT \"Brush & Extra fine\" Marker Komplett-Set finden Sie in unserem Onlineshop. \u2713 Marker mit Qualit\u00e4tstinte auf Alkoholbasis und zwei unterschiedlichen Spitzen. \u2713 Ideal f\u00fcr unterschiedlichste Mal- und Zeichentechniken. \u2713 \u00dcberzeugen Sie sich selbst! \u2794 Jetzt g\u00fcnstig online kaufen.",
  "manufacturerid": "f58f91bac483a6ea628e9d2be7a4b1e1",
  "manufacturer_title": "pebeo",
  "url": "/GRAPH-IT-Brush-Extra-fine-Marker-Komplett-Set-Display-mit-96-Markern.html",
  "maincategory": "9fb3ea945468f7c61b471c7401e7bad4",
  "maincategoryurl": "/Zeichnen-Grafik-Design/",
  "groups": {
    "DISCOUNT80": {
      "price": 2.00,
      "price_no_vat": 1.68
    },
    "DISCOUNT800": {
      "price": 20.00,
      "price_no_vat": 1.68
    }
  },
  "category": [
    {
      "catid": "cb38ac47b6005b97cd1bbed61c70fb18",
      "shopid": [
        "4"
      ],
      "path": "shop/unser-angebot/grafik-design/stifte-marker/copic-layoutmarker/",
      "title": "Layoutmarker"
    }
  ],
  "attributeStr": [
    {
      "id": "76c7e398b67f441fdc9220e11b419945",
      "title": "Strichbreite",
      "value": "0,5 mm"
    }
  ],
  "attributeInt": [],
  "attributeFloat": [],
  "timestamp": "2019-09-27 15:34:42"
}

Field definition

Field	Description
parent	Always the identifier of the parent product
attributeStr[], attributeInt[], attributeFloat[]	All attributes of the current variant Unsorted list of attributes, one entry per value (or values separated with unique delimiter - Makaira separates them during import) depending on the value type into the respective array: String (attributeStr) / Integer (attributeInt) / Float (attributeFloat) for more information see section Attribute Handling

📘
Benefits of attributes, attributeStr, attributeInt and attributeFloat

Variant precise filtering for filter combinations
(T-Shirt XL in black will only be displayed if this combination exists)

Title of attributes can be imported and used afterwards in the frontend
(otherwise has to be done via translations in the frontend)

Automated casting to Int/Float/String without our special suffixes for mapping data

Products without variants (pseudo variants)

If your catalog has no variant concept (each item is a single SKU), you still have to deliver a child document for every product. Makaira does not generate this child automatically for direct NDJSON imports — you have to include it in the file yourself.

📘
Persistence Layer exception
When you push data through the Persistence Layer instead of the NDJSON import, the pseudo variant is generated automatically (id pattern <product_id>_pseudo_variant, with isPseudo: true). This section only applies to direct NDJSON imports.

The pseudo variant is a variant document that mirrors the parent product and adds two markers:

"isPseudo": true — tells Makaira this is a synthetic variant that should be hidden from admin counts and most listings.
"isVariant": true — marks the document as a variant.

We recommend the ID pattern <parent_id>_pseudo_variant so the synthetic ID is recognizable and stable.

{
  "id": "10013859950_pseudo_variant",
  "type": "variant",
  "parent": "10013859950",
  "isVariant": true,
  "isPseudo": true,
  "shop": "4",
  "active": true,
  "onstock": true,
  "stock": 10,
  "ean": "48690A",
  "title": "GRAPH'IT \"Brush & Extra fine\" Marker Komplett-Set",
  "price": 220.50,
  "url": "/GRAPH-IT-Brush-Extra-fine-Marker-Komplett-Set.html",
  "timestamp": "2019-09-27 15:34:42"
}

💡
Recommendation for catalogs without variants
If you have no real variants, the attributes / attributeStr / attributeInt / attributeFloat structure is overhead — there is nothing to differentiate between siblings. For these catalogs we recommend using flat Dynamic Fields on the product (e.g. color_str_short, weight_float, is_new_bool) instead. They are easier to produce in the feed, get the correct Elasticsearch type automatically, and are filterable and sortable out of the box.

Type manufacturer

{
  "id": "f58f91bac483a6ea628e9d2be7a4b1e1",
  "type": "manufacturer",
  "manufacturer_title": "pebeo",
  "timestamp": "2019-09-27 15:34:42",
  "url": "/markenwelt/pebeo-gs/",
  "active": true,
  "shop": "4",
}

Type category

{
  "id": "ffa7bbaff96b15da5579d99247e981bf",
  "type": "category",
  "active": true,
  "hidden": false,
  "depth": 3,
  "sort": 1,
  "category_title": "Tempera + Gouache Farbe",
  "hierarchy": "b5664298b7fb8653356baafc2f8b5664298//baafc2f8b5664298b7fb865762918b32//ffa7bbaff96b15da5579d99247e981bf",
  "subcategories": ["6c52086cb1550335611b759cdf8681bd"],
  "timestamp": "2019-09-27 15:34:42",
  "url": "/farben-hilfsmittel/kuenstlerfarben/tempera-gouache-farbe/",
  "shop": "4"
}

The example above describes a category at depth 3 (Farben & Hilfsmittel → Künstlerfarben → Tempera + Gouache Farbe).

Field definition

Field	Description
`id`	Unique identifier of the category.
`category_title`	Display name of the category.
`active`	`true` if the category should be visible.
`hidden`	`true` to hide the category without deactivating it (overrides `active` for visibility).
`depth`	Level of the category in the tree, starting at `1` for root. Must be consistent with the number of segments in `hierarchy`. All ancestor categories must be delivered as their own documents — Makaira can only build the tree if every node is present.
`sort`	Sort hint within the same depth. Use `1` if you don't need explicit ordering.
`hierarchy`	The breadcrumb path from the root category down to this category, as IDs joined by `//`. The path always ends with the category's own `id`. Examples: a root category has `hierarchy = "<own_id>"`; a depth-2 category has `hierarchy = "<root_id>//<own_id>"`.
`subcategories`	IDs of all descendant categories (children, grandchildren, …), not only direct children. This is used by Makaira at search time to expand a category filter so a request for the parent also returns products in any sub-tree. Leaf categories deliver an empty array.
`url`	Frontend URL of the category page.
`shop`	Shop identifier (multi-shop installations).

❗️
Deliver the full tree
Every category in the path must be exported as its own document. If hierarchy references an ID that has no matching category document, the tree cannot be built and the category will not be filterable.

Custom document types

It is also possible to define own data types by just passing a unique type as string which you can access by the API and display them. Additinal those documents are also searchable in the Makaira search. This can be use for example when you want to have your blog or store locations searchable.

{
  "id": "9gh38eipwgwrwr442",
  "type": "blogentry",
  "active": true,
  "title": "My first blog entry",
  "raw_title": "First entry",  
  "searchkeys": "the,search,keys,of,my,first,blog,entry"
  "timestamp": "2019-09-27 15:34:42",
  "url": "/blog/my-first-blogentry",
  "shop": "1"
  ...
  <your custom attributes>
  ...
}

Field definition

Field	Description
title	Will be searched in the makaira search including wordprocessing like stemming, compound words, etc.
raw_title	Alternative title without word processing only exact matches will hit
searchkeys	Additional words to take into account for search

Attribute handling

To understand the attribute handling we provide an example based on a T-Shirt. Let's assume the T-Shirt has two variants with attributes and some common attributes.

Data type	Title	Attributes	values
product	T-Shirt Emma	Cut* (Id: "987cut"): Cut* (Id: "988cut"):	"V-Neck" "O-Neck" or "A-Neck"
variant	T-Shirt Emma S	Size* (Id: "523size"): Length* (Id: "029length"): Weight* (Id: "736weight"):	"S" 13 2.4
variant	T-Shirt Emma XL	Size* (Id: "523size"): Length* (Id: "029length"): Weight* (Id: "736weight"):	"XL" 25 3.0

For a valid import, the attributes need to be synchronized between the document type product and variant.

For importing these documents in Makaira we always need to have the attributes “synced” in both directions. The variants have to inherit the attributes of the product and the product has to aggregate the attributes of the variants in attributeStr, attributeFloat, attributeInt (based on the type of the attribute).

Also the product (parent of variants) has to aggregate the attributes of all variants grouped by variant in attributes – here the types are not relevant.

The resulting Makaira objects for an import have to look like below.
For better readability we focus in the example on the attribute values.

{
  "type": "product", // or makaira-productgroup
  "title": "T-Shirt Emma",
  "attributeStr": [
    {
      "id": "987cut",
      "title": "Cut",
      "value": "V-Neck"
    },
    {
      "id": "988cut",
      "title": "Cut",
      "value": ["O-Neck", "A-Neck"]
    },
    {
      "id": "523size",
      "title": "Size",
      "value": ["S", "XL"]
    }
  ],
  "attributeInt": [
    {
      "id": "029length",
      "title": "Length",
      "value": [13, 25]
    }
  ],
  "attributeFloat": [
    {
      "id": "736weight",
      "title": "Weight",
      "value": [2.4, 3.0]
    }
  ],
  "attributes": [
    {
      "987cut": "V-Neck",
      "988cut": ["O-Neck", "A-Neck"],
      "523size": "S",
      "029length": 13,
      "736weight": 2.4
    },
    {
      "987cut": "V-Neck",
      "988cut": ["O-Neck", "A-Neck"],
      "523size": "XL",
      "029length": 25,
      "736weight": 3.0
    }
  ]
}

{
  "type": "variant", // or makaira-product
  "title": "T-Shirt Emma S",
  "attributeStr": [
    {
      "id": "987cut",
      "title": "Cut",
      "value": "V-Neck"
    },
    {
      "id": "988cut",
      "title": "Cut",
      "value": ["O-Neck", "A-Neck"]
    },
    {
      "id": "523size",
      "title": "Size",
      "value": "S"
    }
  ],
  "attributeInt": [
    {
      "id": "029length",
      "title": "Length",
      "value": 13
    }
  ],
  "attributeFloat": [
    {
      "id": "736weight",
      "title": "Weight",
      "value": 2.4
    }
  ]
}

{
  "type": "variant", // or makaira-product
  "title": "T-Shirt Emma XL",
  "attributeStr": [
    {
      "id": "987cut",
      "title": "Cut",
      "value": "V-Neck"
    },
    {
      "id": "988cut",
      "title": "Cut",
      "value": ["O-Neck", "A-Neck"]
    },
    {
      "id": "523size",
      "title": "Size",
      "value": "XL"
    }
  ],
  "attributeInt": [
    {
      "id": "029length",
      "title": "Length",
      "value": 25
    }
  ],
  "attributeFloat": [
    {
      "id": "736weight",
      "title": "Weight",
      "value": 3.0
    }
  ]
}

Group-specific overrides (B2B / customer groups / discount tiers)

The groups field lets you ship per-customer-group values for a document — typical use cases are B2B pricing, discount tiers, or hiding products from specific customer groups. At request time, Makaira applies the overrides when the API call sets the query.group constraint.

{
  // Other fields
  "active": true,
  "price": 99.99,
  "groups": {
    "wholesale": {
      "price": 79.99,
      "active": true
    },
    "vip": {
      "price": 69.99
    },
    "internal": {
      "active": false
    }
  }
}

Group names (wholesale, vip, internal in the example) must match the value sent in the API request as query.group and are case-sensitive.

🚧
Requirements for the feed

The groups field has to exist on the product and on all of its variants (including pseudo variants).

All groups inside the object should expose the same set of sub-fields, otherwise the Elasticsearch mapping breaks.

Group overrides only apply to flat fields. Nested structures like attributeStr / attributeInt / attributeFloat / attributes cannot be overridden per group.

❗️
Visibility fields are gated, not toggled
The visibility fields active, searchable, hideOnLists, and onstock cannot be unlocked through a group override. The root value acts as a hard gate; a group override can only further restrict visibility for that group. So setting root active: false and groups.<x>.active: true will not make the product visible for group <x>.

For the full feature description, more use-case examples, the precise filter logic, and all limitations see B2B / Customer Groups.

Revision based import

If you want to import your product data in a continuous and incremental manner over time you can provide us with a public endpoint in your infrastructure where we can pull data from. This data is polled by our importer process. For this, you need to fulfill a specific API Specification described below.

The data are pulled on a revision base - so your system has to somehow keep track of the changes in your database so that you can always answer the question in your system "what has been changed lately". For more information on this check section Fundamentals -> Revision

Fundamentals

Revision

A revision describes a version of a document.
Revision numbers are integers.
No revision number should be used twice.
Each change (create, update, delete) on a document (products, variants, categories, etc.) in your system should create a new revision number.

Poll Principle

We are pulling the data out of your system by sending you a specific POST request to a given endpoint
Your endpoint has to be public, but can be secured by an extra BasicAuth.
Your endpoint has to validate our call by checking the signature (see next section "HMAC Signature")
We are always pulling a batch of data starting from our maximum known revision number in Makaira.
If your system doesn't send back data (maximum revision number reached) or throws an error, we are reducing the polling speed incremental from 1 second (fastest poll) down to 60 seconds.

HMAC Signature

The endpoint can be secured by a BasicAuth but it must at least implement the check of the HMAC Signature to hinder unauthorized data access of your systems data.
HMAC Signature is calculated by using the customer unique Makaira Secret as a signature key and the combination nonce: request-bodyas value for the signature.
The Nonce will be sent in the header X-Makaira-Nonce and normally contains the current unix timestamp.
The Signature will be sent in the header X-Makaira-Hash from the Makaira systems.

Example Code to generate the hash in JS (e.g. in Postman)

var nonce = postman.getEnvironmentVariable("nonce");
var secret = postman.getEnvironmentVariable("secret");

var hashString = nonce + ':' + request.data;
var hash = CryptoJS.HmacSHA256(hashString, secret);

Actions

The polling endpoint in your system has to support all three different actions for the correct dataflow:

getUpdates

This is the main endpoint that we need to import the data from your systems. We are asking your system to send us back a number of items and the revision number to start from (all documents with a revision number HIGHER than the one we are sending you)

POST <your-polling-endpoint>

Header X-Makaira-Hash: <our calculated hash>
Header X-Makaira-Nonce: <our random nonce to calculate hash>

{
    "action": "getUpdates", 
    "since": 253425, // can be -1 at minimum if we build an makaira index from scratch
    "count": 100, // normally between 1 and 500, max can be configured higher if needed
    "language": "de" // one of the languages you provided us in "listLanguages"
}

We are expecting the following answer from your endpoint where as the data has to follow the structure below product data type

Response:

{
    "language": "de", // the requested language again
    "highLoad": false, // you can send us a true here if your systems are under heavy load and we should reduce polling speed
    "count": 1, // the count of documents you are sending us
    "changes": [ // the array of documents with a small header (id, type, deleted, sequence) and the "data" that will be imported
        {
            "id": "product_-1110", // the ID of your document
            "type": "product", // the type of your document
            "sequence": 8159, // the revision number of your document
            "deleted": false, // can be true if document was deleted - in this case data can be an empty object.
            "data": {
                // full document data
                
            }
        }
       // ... more objects up until our requested count 
    ]
}

listLanguages

To set up the import process correctly our importer needs to know how many languages are supported by your shop.

POST <your-polling-endpoint>

Header X-Makaira-Hash: <our calculated hash>
Header X-Makaira-Nonce: <our random nonce to calculate hash>

{
    "action": "listLanguages"
}

We are expecting an array of ISO-639-1 language codes as a response e.g.:

Response:

[
    "de",
    "en",
    "fr",
    "nl"
]

getReplicationStatus

To check what the maximum revision number is (e.g. for displaying the "not yet imported/open changes" on the dashboard) we need the following action supported by your system:

POST <your-polling-endpoint>

Header X-Makaira-Hash: <our calculated hash>
Header X-Makaira-Nonce: <our random nonce to calculate hash>

{
  "action": "getReplicationStatus",
  "indices": [
    {
      //...
      "language": "en",
      "lastRevision": 266, // the maximum revision that we imported until now
      "openChanges": null
      //...
    },
    {
      //...
      "language": "de",
      "lastRevision": 156, // the maximum revision that we imported until now
      "openChanges": null
      //...
    }
  ]
}

We are expecting the indices array where the openChanges key was filled with the appropriate calculation of "how many documents are left to import after lastRevision"

Response:

{
  "indices": [
    {
      //...
      "language": "en",
      "lastRevision": 266, // the maximum revision that we imported until now
      "openChanges": 20 // <-- this key was filled by your system
      //...
    },
    {
      //...
      "language": "de",
      "lastRevision": 156, // the maximum revision that we imported until now
      "openChanges": 130 // <-- this key was filled by your system
      //...
    }
  ]
}