Overview of the Data Migration Process – Populi Knowledge Base

Here's an overview of the major points of the Populi data migration process. The finer points may vary from school to school, but all implementations follow this storyline. If you have any questions about this, don't hesitate to contact your Populi Account Manager.

Setup during migration

During data migration, there are certain setups steps you can perform in Populi that are outlined in this article. We encourage you to begin this setup as much as you're able—not only will it get you up-and-running faster, it will also help you get familiar with Populi and give you a better understanding of things during the data testing phase (covered in detail below).

Initial account setup

On the initial account setup form, you'll indicate whether you wish to migrate data from your legacy system into Populi. Depending on its quality, we are able to migrate the following data from your legacy system(s) into Populi (subject, of course, to the Terms of Service):

Academic Data: Course catalog, programs and degrees, current and historic transcripts, together with related data explicitly agreed to by Us and You. This constitutes the majority of the academic data exported by most systems and includes things like grade scales, but does not populate things like degree audits—that will require manual setup on your part.
Financial Data: Populi does not import historic financial details. All outstanding student balances and shall be imported as follows: Positive balances shall be associated with one “Balance Forward” fee. Negative balances shall be associated with one “Balance Forward” credit. Such fees and credits shall not contain specific component or payment details.
Library Data: Library resource data in MaRC format (which is converted into Dublin Core for use in Populi Library).
Person Data: Basic personal and contact information for students, faculty, staff, admissions leads, and other people stored in your system.
Donations/Donor Data: Donor information, donation info (amount, fund, campaign, payment method, custom info, etc).
Admissions Data: Lead names and contact information.

If your school is not migrating data from a legacy system, then your Populi site is ready to go as soon as you complete the signup form. The next thing you'll want to look at is our guide to setting up different areas of Populi for schools without data import. But if you are migrating data, keep reading!

Getting your data to us

If we're importing legacy data, after completing the initial setup steps described here, you'll need to get your data to us so we can begin converting it for use in Populi. At any given time, we'll be migrating data for several different schools; the sooner we get our hands on your data, the quicker you secure your place in the migration queue. Your implementation manager will schedule a "kickoff" phone call to discuss your needs and come up with a setup plan that takes into account both live site setup and data migration. Depending on the migration queue and how quickly you'd like to begin using Populi, we generally encourage you to do some live site setup that the migration process might otherwise have accomplished.

The most secure, straightforward way to transfer your data to us is by using the Upload feature in your Populi Account section.

Get aholt of your legacy database. For Microsoft Access databases, send the MDB or .ACCDB file. For MS SQL databases, send the .BAK file. If you have questions about data formatting and what we need on our end, please contact Populi customer support.
Once you have the database files, compress them using a ZIP program.
If you can, give the ZIP file a name indicating the contents. For example, something like Yourcollege Complete Academic Backup [DATE].
Log in to your live Populi site and go to the Account section.
Click the Files view.
Under Legacy Data Import Files, click Upload Files.
Upload the files.
Click Done when you're finished.

During the data testing phase (see below), you may need to repeat the above steps as we refine our import scripts and test them against the most current version of your database.

Additionally, we will need examples of student transcripts generated by your legacy system. In any system, transcripts are generated using a combination of a student’s historical academic information and your policies for constructing GPAs. A student's historical academic data is not enough to determine how to generate his transcript. The transcripts you provide help us understand how you construct GPAs, serving as an "answer key" when we compare them with the transcripts generated by the testing data in Populi. Accordingly, we'll need to see transcripts that show:

A current student
A student no longer at the school
Transcripts with good grades and bad ones
Examples with pass-fail (P/F) courses, if applicable
A course that was initially failed and then passed on a retake
Student honors, if applicable
Transfer credit

Initial migration and data testing

Depending on the format and condition of your data, the migration and data testing (a process called "validation") generally takes from four to sixteen weeks. During this time you can continue using your existing system as usual.

The validation process begins when a Populi Data Manager writes a script to translate your data into Populi's database structure. The transcripts you provide are crucial for helping refine the scripts. The DM will then upload that translated data into a test instance of Populi called a validation site. After some initial checks, the validation site will be released to you for testing. You (and others at your school) will examine the data and flag any errors for the DM so we can further refine the scripts. Some notes:

Expect at least a few rounds of testing and refining. The process depends on the condition and complexity of your previous system, how aggressively you test the data, and how many problems are uncovered and reported.
Get a team of people together to perform the review. Make sure they're familiar with your previous system and database. No one knows your data better than you, so make sure to devote staff time to this effort!
- This article describes how to add new people to Populi. You'll also want to consider which user roles you'll assign each person.
- This article describes how to set up user accounts.
- Make sure you add people and set up user accounts in your live site (see below); we will copy them over to your validation site.
- If you're using an Identity Provider other than Populi for your live site, contact Populi Support for assistance with duplicating the user accounts for your validation site.
Compare transcripts from your old system to the new ones generated in Populi. Common issues include grade scales, transfer credits, retakes, and any unusual academic situations.
You do not need to examine every single transcript in detail! At this point, data issues will be systemic, so you need only look at some representative transcripts covering the scenarios laid out above. For example, to test how transfer credits have been migrated, you need only look at a couple transcripts with transfer credits. To test P/F courses, just look at a few such transcripts. And so on.
Also examine things like contact information and any custom fields we've imported.
When you find and report problems, the DM will be able to improve the script and fix all such problems in your database.
Populi validation sites do not store the changes you make. If you find problems, fixing them in Populi won't help us, nor will it correct any underlying issues... nor will those changes make it into your finalized database! As we improve the scripts, the validation site will be refreshed with the newest changes and fixes.

Live vs. Validation sites

Here are some notes about the validation site vs. your school's "live" site, which are two different sites.

The validation site exists only temporarily for the purposes of checking how your migrated data appears and behaves in Populi; it disappears once your data is migrated. Its URL will be something like yourschool-validation.populi.co. Your school's live site will be what you end up using; it will have a URL like yourschool.populiweb.com.
Your live site login will also get you into the validation site.
The validation site will be periodically overwritten as the data import scripts are refined (see below). Your live site will not be affected—you can start setting up Populi for your school.

Final import and going live

When you are ready to sign off on the migrated data, weʼll schedule a date to "go live"—that is, to push the finished data to your live site. The process works as follows:

If you've continued using your legacy system during migration, just before the scheduled data, you'll retrieve the most recent backup data from your old system.
You'll transfer that legacy database to us via Account > Files (see above).
We use the refined import script to move the data into your Populi live site—not your validation site! Your live site will go down for around half an hour while we fill in the data.
Once that's done, your live site will contain:
- Anything you've set up in your live site during data migration. For example, you might have set up degree audits, added some new catalog courses, and started work on admissions applications.
- The data from your legacy system, now migrated into Populi's database structure and ready for your use.
- All of your data will now function seamlessly within Populi—you'll be able to, for example, run a degree audit for a student who's been migrated over from your last system, or add a graduate application to someone else who was imported...
We shut down your validation site.
In rare cases, you may discover errors with the imported data after it's live. If this happens, just contact Populi Support.

If you've been using your old system up until this time, at this point you should stop using it (unless you want to double-enter it in two systems). You'll now want to complete Populi setup and start rolling it out for use by everyone at your school.