MongoDB

1. What is MongoDB?

MongoDB is a document-oriented NoSQL database used for high volume data storage. Instead of using tables and rows as in the traditional relational databases, MongoDB makes use of collections and documents. Documents consist of key-value pairs, the basic unit of data in MongoDB. Collections contain sets of documents and functions, which is the equivalent of relational database tables.

2. Add MongoDB Data Source

1. Click Add Data Source

2. In the menu, select MongoDB

3. Config information of Data Source

• Display name: name of your MongoDB source

• Host: the IP address where your database is hosted

• Port: you need to check with your system administrator to choose the port opening for connecting

• Username: username of the MongoDB user you created

• Password: password of the MongoDB user you created

• Enable TLS/SSL Config

4. Test Connection

After inputting the Config information, click Test Connection to check whether your MongoDB source connects successfully.

5. Click Add to add Data Source

Your MongoDB will be displayed in the Data Sources (e.g. sample mongodb) with the information of the Owner, Type, Date Modified, and Action.

3. Pull data from MongoDB Data Source

1. Click the Jobs tab then click Add Job

2. Config information for Job

• Job name: name of your job

• From Data Source: select your data source to add the job

• From database: select one database suggested in the data source. If the database you need doesn’t appear on the list, please type exactly the database you want.

• From table: select one table suggested in the database. If the table you need doesn’t appear on the list, please type exactly the table you want. Or you can choose to sync all tables.

• Sync to data warehouse: toggle this to turn syncing to the Data warehouse on or off. You can create a new database or pick up the existing database on the Data warehouse to sync.

• Create new database: choose one database suggested in the data source or you can create a new one

• Create new table: choose one table suggested in the database or you can create a new one (also when you create a new database)

• Sync to data lake: toggle this to turn to sync to Data lake on or off

• Sync Mode: you choose between Full Sync and Incremental sync:

• Full sync: sync everything in the data source with the designated destination, data lake, data warehouse, or both

• Incremental sync: transfers changed all record and catalog data since the last synchronization (full or incremental)

• Scheduler type: choose the type you want to sync

• Run only once: only runs one time at a designated Date and Time

• Recurring: set up the repeated sync using the time scheduler

• Frequency: select the time interval between the data sync

3. Click Add Job

4. Check your synced job: Synced Jobs will be displayed in Jobs with the information of their mode, status, date synced, and action. You can click Force Sync to sync manually to ensure you access the up-to-date data.

5. Click the Jobs History to see what jobs have been done in the system so far. You can track all the activities of the jobs, including the time, the number of occurrences, and how many columns each job has generated.

4. Demo

This is a demo of the entire process of adding MongoDB Data Source.

We hope that the above instructions will be useful to you. Let's have a look at our other articles to learn more about adding other databases with Data Insider.