Directory Connect – Populi Knowledge Base

Directory Connect lets you subscribe to webhook notifications from Populi into your own network environment when the following events occur:

A user is created
A user is deleted
A user's personal information (name, address, phone, email, etc.) is updated
A user changes their password

Think very carefully before linking a production site to a Populi validation site through Directory Connect! Every time you change data on your validation site, it will be fed into your production site and could end up overwriting active user account credentials or filling it with other junk data that may pose a security risk. There are many valid reasons for connecting production sites with your Populi validation site, and so we do not prohibit it, but if you do so, do so with open eyes!

All relevant information describing the user is included in each webhook notification; in the case of a password change, the new password is also specified. These notifications enable you to replicate changes to user identities from Populi into your own private directory systems like Active Directory or LDAP.

Because notification messages contain highly sensitive information such as user passwords, Directory Connect relies on HTTPS communication protocols in addition to OpenSSL public-key cryptography.

How to configure Directory Connect

1. Generate an RSA private key

Generate an RSA private key to use for decrypting user passwords. Using the "openssl" toolkit (downloadable for Windows, preloaded with every other common OS), this can be done by executing the following command to write a new key to the file "private.pem":

openssl genrsa -out private.pem 2048

Because of its cryptographic sensitivity, ensure that your private key file is restricted with read-only access to as few users as possible.

2. Generate a matching RSA public key

Generate a matching RSA public key which can be distributed to Populi for encrypting user passwords.

openssl rsa -in private.pem -pubout -out public.pem

This will write your RSA public key to the file "public.pem" in PEM format, like the following example:

You may share your public key with the appropriate people; Populi will use it to encrypt data that you can later decrypt using your private key.

3. Set up an HTTPS endpoint

Build and expose an HTTPS web service endpoint. To receive webhook notifications from Populi, it must be reachable from the public Internet (where Populi's servers are located). This service needs to be available at a single URL (with a valid SSL certificate chain), which will be invoked once for each notification.

4. Configure the Directory Connect integration in Populi

Go to Account > General Settings > Integrations. In the Directory Connect setting, provide the URL for your web service and the public key you generated in Step 2 to use for password encryption.

WARNING: As soon as you save this setting, as long as the Directory Connect integration remains enabled, Populi will capture production user events and transmit them as notifications to your web service. Ensure that your service's logging and security policies are appropriate for maintaining privacy of this sensitive data.

When a notification is received and processed correctly, your service is expected to acknowledge with an HTTP response code in the 200's (200 OK, 202 Accepted, etc.) at which point the event will be discarded from our servers. (Discarded messages cannot be retrieved.) Any other response code will be treated as an error, and the notification will remain queued for delivery to your web service. Delivery of queued notifications will be retried periodically until they succeed.

Disabling or deleting the Directory Connect integration will discard all notification messages in the queue for delivery.

5. Test your connection

Click the Test button to test the connection and your web service. This will generate a series of notification messages based on your own active Populi user and ensure that they are received and processed correctly. Any failures will be reported to you for troubleshooting.

About Notifications

Notifications are always delivered in FIFO order according to the time the corresponding event took place.
In the case that event notifications cannot be delivered for an extended period of time, Populi servers will maintain a queue awaiting delivery of the most recent 1000 undelivered notifications. Once your service resumes acknowledging messages, all pending undelivered notifications will be delivered immediately. (If the number of undelivered notifications exceeds 1000, the oldest notifications will be discarded.)
If you decide to discontinue your use of Directory Connect, please disable the integration from your account settings to avoid accumulating undelivered notifications.

Notification message format

Notification messages enclose JSON-structured data resembling this example:

{
  "event": "PASSWORD_CHANGED",
  "timestamp": "2020-01-27T10:37:54-08:00",
  "user": {
    "object": "person",
    "id": 12345,
    "first_name": "Count",
    "last_name": "Chocula",
    "middle_name": "",
    "prefix": null,
    "suffix": null,
    "preferred_name": null,
    "display_name": "Count Chocula",
    "gender": "dude",
    "image_id": 0,
    "birth_date": null,
    "status": "active",
    "alien_registration_number": null,
    "home_city": null,
    "home_state": null,
    "home_country": null,
    "former_name": null,
    "license_plate": null,
    "bio": null,
    "updated_at": "2020-01-27T18:37:54+00:00",
    "hispanic_latino": false,
    "ipeds_race_id": null,
    "resident_alien": null,
    "localization_id": 1,
    "added_at": "2019-06-11T17:42:58+00:00",
    "added_by_id": 123,
    "private_profile": false,
    "citizenships": null,
    "addresses": [
      {
        "object": "address",
        "id": 111,
        "street": "123 Main St",
        "city": "Busytown",
        "state": "CA",
        "postal": "98765",
        "country": "USA",
        "address_type": "home",
        "is_primary": true,
        "public": true,
        "added_at": "2019-06-14T21:56:30+00:00",
        "added_by_id": 12345
      }
    ],
    "phone_numbers": [
      {
        "object": "phone_number",
        "id": 222,
        "number": "(123) 456-7890",
        "phone_type": "home",
        "is_primary": true,
        "ext": null,
        "public": true,
        "added_at": "2019-09-12T22:02:09+00:00",
        "added_by_id": 12345
      }
    ],
    "email_addresses": [
      {
        "object": "email_address",
        "id": 333,
        "email": "count.chocula@diabeetus.com",
        "email_type": "home",
        "is_primary": true,
        "public": false,
        "delivery_problem": "",
        "delivery_problem_at": null,
        "delivery_problem_info": null,
        "verified": false,
        "added_at": "2019-06-11T17:43:18+00:00",
        "added_by_id": 12345
      }
    ],
    "username": "countchocula"
  },
  "test_mode": false,
  "new_password": "bCV+qsKITk98NOf63eVzfywUynPiRl15IkZ9o8daUsSwk2ES8H0LiGNzDaOOXcB2OXQL/3X1go0V/Pvwig9NzawvmGGAVDW2saDlxT4Vnw8sGTcOAIF0WWyewJmB8IPJ09TXwc0qLOyHbmwTgEAlRF9+k/8cGvqQFZ1iJsx8eKg26IdXRAvnas/YYr8Xg/eoiAlmlH0TYOW5VWCd7RgtElgFrVXY/dU+GL8Eu1jPMpdxPSOvaD3KMqNyqFfkRIvuzwgX1FKY8SQBf8LexNwzrOhR5gpKRNYvs3Nr+AIpXrkMKBkflRa4Y5xJQ0tzplzrTvoEVVVpXuZ/78+TDdpP6WbL8q9PUm6jne2bf7a1FmHL6/AzsQVjvy9kCwj1XbjyXQN9BrRLc6ecYLyTRZwrhSYexZHtC8MbbQTsHZ1FLm2ZGP6FehvcZ68QO92slKtQu9eBEhnLZlt2Ier2h0hTOg2TuNDcB1Tw+vfQxq/M9cfU+ClBAnrVXp4E3ws2Nl9avswKRVfGpoh84u+9I2uYg2Wt1Izu/+N8KfqwMOmb6UpszXn5i6M0vCAjBnswU6cRj6GZgAVnVaOZs4b480nZLa3oKzEOATcxTw7jnIUezitHhCSclkya5D6PuYA+mmIXnM4uPXfMOMDqaIxc13NGUvjwpwjKJY3+yfNPKYcNvRw="
}

A few notes:

The eventfield will be one of: USER_CREATED, USER_UPDATED, USER_DELETED, PASSWORD_CHANGED
All timestamps are encoded using the standard RFC3339 format.
The new_passwordfield is absent in all notification messages except for PASSWORD_CHANGED.
The test_modefield indicates when the message was generated using the manual Test function in the Integrations > Directory Connect setting. These messages are useful for confirming your ability to parse metadata and passwords, but should be prevented from changing any active data within your own directory service.

Password Decryption

As stated earlier, user passwords are encrypted for transmission using your provided RSA public key. They are encrypted using OAEP padding, and are base64-encoded to limit their character set to that which can be transmitted in JSON data structures. You will receive encoded passwords in a form like the following:

N3HNl0z0qiGglc+TWKY5/0/8maf89+vFdMv+tOm8bceh3tLuKZYdkvP/FR8uul7VojKY6olvGzdxqbAT76XnhUroibqB2EhVvGIjpWYcYCrmLgpUAXH6BEnpwJA4HI8MdEsYn43oj/SxmecgDzU6cCisUtzVAnrpWpUw0AX1gb1k/SBxUFT/5r9csCvrsQjF2Yiq53diXLLoFv7VqzKu9yZHmRKn2Q2JE2B5NC3VaceQ1z6Ol+6+HuQR7q9TOFLmolb3FKp3RK/Ksk5hj1cR9ILByxmdnZh1wOVInNsObT+qrVnAC4lc5YvHk23QP3zBO4OERY9wpM+oyQT62wXhmgoU6SgQv0OOqLiFFTKJ+S9bpjFc0hUZ7mudsM3LdV0Dp12wWhUVn5AjrOUqIBwINJzLVDV7oTIiSYtq7ziQdDdKB83EE6lFiUXRKEAChmEP46g7vNkLRPJHPsZ//O/q2dqhMnhWU8BhRO6+0bRhEeoqfNCSkte1N0zqQggaDCYyBLbjOw1/vtgvSlNWM6rwVbUoUPLMjklkgKi/KCGloV4Rf7VaD9Fr5D8gOCsDq7mfnsAjpiYscggGBbRsy24GpDBupNxJvCJWQjMTi+MdqOUqRMZg+WHc6S+YJ3N6Fs5tz7vZK1HUwqT2oV5KWlg8PSlTnaHiQNSRCwlbzvCWAEo=

If saved to a file "passwd.txt", this can be decrypted using the following commands (with access to your "private.pem" key file):

openssl enc -base64 -A -d -in passwd.txt | openssl rsautl -decrypt -inkey private.pem -oaep

Popular programming languages typically include OpenSSL interfaces to accomplish this same decryption, such as PHP's openssl_private_decrypt()function.

Note that the example passwords provided in this article may have been injected with nonsense by the documentation guy and are only for demo purposes. In order to test decryption, you will need to use passwords from Populi notifications encrypted using your own public key.

Message verification

By hosting your own web service and acting on inbound JSON-structured data, your private directory system is introduced to a new security risk. A malicious user could independently send their own messages to the same URL used by Populi, with the effect of generating users or changing others' passwords in your system. For this reason, it is critically important to verify that incoming messages actually originated from Populi's servers. By ignoring any messages that cannot prove their authenticity, you maintain the security of your private directory system.

Every notification published by Directory Connect will include two HTTP headers for the purpose of verification:

Populi-RSA-SHA256-Signaturecontains a RSASSA-PKCS1-V1_5 formatted signature of the message body digest (hashed using the sha256 algorithm), made using a RSA private key owned by Populi. This hex-encoded signature can be verified using the matching public key to ensure that the content of the message is authentic and has not been tampered with in any way.
Populi-RSA-Public-Key-Fingerprintcontains a fingerprint identifying, for your reference, which public key should be used to verify the signature.

At this time, Populi encrypts notification messages using a single keypair, the fingerprint for which is 7b:96:d1:cb:15:04:5d:d0:45:fa:5f:96:d7:25:c1:f6

Its public key contents are as follows:

In the future, Populi may offer additional public keys (identified by their fingerprints) for signature verification or retire previously used keys.

If the HTTP header "Populi-RSA-SHA256-Signature" is saved to a file "signature.hex", you can verify the contents of its associated "message.json" (the body of the HTTP request) using the following commands, with Populi's public key saved as "public.pem":

xxd -r -p signature.hex signature.bin
openssl dgst -sha256 -verify public.pem -signature signature.bin message.json

The "xxd" tool (used above for hex-to-binary conversion) is included with most Linux distributions and macOS, and can be installed for Windows as part of the free-to-download Cygwin package.

Note that, if you use the "openssl" command as in the example above, the contents of the "message.json" file must match exactly with its binary form when it was signed. Common text editors like "vi" may encode a trailing newline at the end of the file, which would prevent signature verification.

Standard programming languages offer interfaces to OpenSSL to simplify this signature verification and hex-to-binary conversion, such as the functions openssl_verify()and hex2bin()in PHP.