Custom Source

Scenario 1: new source writing to existing models

Example use case: you store incidents in a home grown system.

Steps:

  1. Develop a new Airbyte source connector that will extract data from the new source, in this case your homegrown incident management system.
  2. Fork the Airbyte Faros Destination and add a new converter for your source, which in our example use case will write against our ims_Incident model (among others).
  3. Once both are published, add your new source and modified Faros Destination in the Airbyte UI , in Settings > Sources > + New Connector and Settings > Destination > + New Connector respectively.
  4. Create a source and a destination from those connectors, and link them in a connection. Remember the πŸš€ Faros Destination Table Prefix convention.

Note that unless you get your Faros Destination changes merged upstream, you will be responsible for maintaining your fork.

Scenario 2: extended source writing to existing models

Example use case: add support for GitHub Actions in the GitHub source

This is similar to scenario 1, except that you fork the source connector as well. In this example, the Faros Schema already has the necessary models (cicd_Build among others).

Unless your changes in the source or Faros destination connectors are merged upstream, you will be responsible for maintaining those forks.

Scenario 3: new / extended source writing to new / extended models

Example use case: track EC2 instance usage.

This is similar to scenario 2, except that you use a πŸ—Ί Custom Schema.