In our Importing your data article, we explained that there are two main ways that you can import data to CiviPlus; one simple and one more advanced. If you haven't already, we recommend to read the Importing your data article fully before the one below, to ensure you have understood the simpler import tools fully and the benefits they offer before deciding to proceed with more advanced tooling.

This article is designed to support you in your use of the advanced import tools that CiviPlus offers. Accompanying this guide are a number of short accompanying videos that demonstrate the preparation and imports of an example set of files that contain test data.

Similarly to working with the simple import tools for CiviPlus, for each advanced import you plan to run you will need to prepare a text file containing the data that you want to import. This needs to be in the Comma Separated Value format (CSV) - each row representing a different record to be imported, and each column (separated by commas) representing a different field that you would like to import values against. It often makes most sense to format your data first in Excel or Google Sheets before exporting to a CSV format, including a row showing the heading for each column. It really helps the import process if you make sure that the name of the column heading matches the name of the field in CiviPlus.

The advanced CiviPlus import tool is called the API CSV Import tool. You can locate it here in your administer menu:

This importer tool sends CSV data to the CiviPlus database via API and for any

entity (database table) in CiviPlus. From this user interface you can upload a file; you can tell CiviPlus what you’re importing and which field in CiviPlus matches which column in your CSV file:

If CiviPlus cannot accept a row in your CSV file for any reason, the API CSV import tool produces an error file to show you exactly where the import issue lies. Before you leave the import tool screen, you should download the error file (it won’t be accessible again after you navigate away).

You can read the errors, make adjustments to your CSV files and run an import again with valid data. We recommend saving any error file so you can easily find it again. When you try the API CSV import tool for the first time we recommend importing small sets of data so you can become familiar with how it works. Having just two or three rows of data in an initial test import is plenty.

If you have a lot of data of different types to import, it can feel like a big task ahead to overcome. We have put together a handy set of tips and and videos below to help you plan your migration.

CiviPlus’s database stores each different type of record in its own data

table (entity).

Through the CiviPlus user interface you are able to easily join different types of data together in a number of ways - for example, by connecting your contacts with their personal data, other custom data you store about them, activities they have been involved in, events they have attended, memberships they hold and more. However, when importing data to CiviPlus with the API CSV Import tool it is important to understand that these different types of data should be imported separately.

Your data might contain most, if not all, of the different entities below:

Ensure that you import each entity above via a separate CSV file. The order in which you import entities is important too. Of course, there’s no point trying to import a membership if the contact record for the individual member is not in the system yet! We recommend the following import order:

Of course, you can skip any entities that aren’t relevant to you yet.

If you aren't sure which fields are available or should be included for the entity you are looking to import, the import tool is able to help. By progressing to the second step in the import tool ("Match Fields")with an example file for your chosen entity, you will be able to see all the available fields for that entity type in your system:

If you are still working on preparing your files and just needed to check the available field names, you don't need to progress beyond this point until you are ready to re-start the import tool with your finalised file.

External Identifier is a useful field on every contact record and we recommend

including it in most CSV import files.

When importing new contact records for the first time, give each contact a unique External ID - either a brand new value or the contact’s ID from your legacy system. When importing other data associated with contacts,

you should reference the same External ID you gave the contact in a column so that CiviPlus can accurately link data to the contact it is relevant to.

We explore this more in the series of videos provided later in this article, but you can also see an example in the spreadsheet snippet below:

If you have a large data set, either with many rows of data or many columns of

fields, it can be worth splitting one CSV file into “Part 1”, “Part 2” etc. This

minimises upload times and makes troubleshooting and mitigating any errors more manageable. As a rule of thumb: have fewer than 5000 cells in your CSV file.

If your source data exists across multiple sources (for example several different spreadsheets, or a legacy CRM and also an email marketing tool), if you are able to then it can be helpful to combine contact lists before importing them. This helps you ensure all field names and formats are consistent but is also an opportunity to spot and remove duplicate contact records. We do this in Video 1 - Combining Contact Lists later on in this article.

Combining contact lists doesn't mean losing sight of where the data originally came from; you can include a “Source” column to show this.

Ensure that every field in your dataset - ie, every column in your CSV file - matches a field in CiviPlus.

If you haven’t already, create dummy records in CiviPlus to understand which

fields come with each entity out-of-the-box. Map as many of the columns in the data you plan to import as possible to these core CiviPlus fields.

When working with the advanced import tool, we recommend that you complete a mapping exercise for all your existing data points before your first import so that you have a clear plan that you are following.

Pick a format for your mapping document that allows you to easily share it. It can be a vital resource to the Compuco team or anyone else who might ultimately need to understand your data in future.

Even a simple table or a spreadsheet can help you keep track of the old and new field labels:

In CiviPlus you can have different types of contacts. Particular sets of fields can

be associated with one contact type but not available to be used with another. For example, you might want to capture “Favourite Colour” for Individual contacts only and “VAT number” for Organisations contacts only. Neither of these fields would make sense with the other contact type.

Similarly, one contact can have different types of addresses on their record,

different phone numbers listed for them, and different types of relationships with other contacts. As a rule of thumb:

To visualise this better, see Videos 7, 8, and 9 below.

You can view CiviPlus’s default types and add any other types that you need from the Administer menu:

Import values should be formatted consistently and in a way that matches configurations in CiviPlus. In particular, when working with the CSV API import tool, it is important to ensure that:

After any file import, you should check to see how the imported records look within CiviPlus to confirm that the data was imported as you expected.

A useful way to do this is to include at least one contact in your import file that has a value for every column in your existing data set, ie a 'Golden Record'. If that contact doesn’t exist, we recommend to create one or two for test purposes. This would then be the first record that you check after an import; if something doesn't look right on this record after your file has imported, then there is a very high likelihood it also won't look right on other records.