Skip to main content

What’s new with the (Corporate) Data Team?

Posted by: , Posted on: - Categories: Analytics, Collaboration culture, Continuous iteration, Data and search, Experts in what we do

Some readers might know who we are, but I’m sure that many still don’t know. So, this post aims to answer a few existential questions about the (Corporate) Data team that you may have.

So, where did you come from and where are you now?

The Data Integration Team (DIT) was originally part of the Corporate Applications Team.

However, following some organisational changes, we moved to Data and Search, under the directorate of Digital Development, with “other Data Teams” and Development teams, which also makes sense as we are data integration developers.

Who are you?

Well, currently there are 3 members in this team: David, Noel and me, Matias (aka Matt aka Mat aka The-Argie-that-won’t-shut-up-and-sometimes-I-struggle-to-get-his-jokes-but-brings-toffee-filled-cake-so-it’s-fine-also-where’s-my-slice [1]).

That’s fine, however what do you do?

We do so many things, but primarily we design, develop, test, deploy, and maintain integrations for different types of data. There’s a broad menu including:

  • people data: combine data from different sources into a single harmonised version of truth
  • financial data: perform direct transformations of financial data from different sources to translate and send it to Agresso (Financial features – Commons)
  • organisational data: merge organisational information into a single organisational structure
  • space data: store and provide space information to other systems
  • telephone and location data: store and match with people information and space information
  • asset data: perform transformation on the data, and pass it to subscribing systems

In more simple terms, we allow different systems that contain different types of data to share it.

Some of these translations of data happen on the fly and the results are not stored, just passed through; for instance: finance data. There’s a backup of the file that was processed just in case something fails but not stored in a database to share it with other systems.

In other cases, when we do store the data, we do it in a normalised version, so that the data that comes from different systems that have different structures is molded into a single structure which is the same for all.

This is the case mainly for People Data. We currently have:

  • 6 sources for people data: HAIS (Agresso), HAISL (ResourceLink), Pass System, MNIS (both Lords and Commons data), PAD and AD
    • HAIS, HAISL, Pass System and MNIS can create People Records
    • if the record already exists they add more info to it
  • PAD and AD add more information to the record: Telephone numbers, location and network user information
  • Archibus provides space data that is also on PAD, so from PAD we know the people location information (who is where), but the description of the space (the “where” information) comes from Archibus.

Ok, that makes sense, but how are you doing?

To measure success, we introduced metrics. We have been analysing these metrics for a few months now, and we added a few right before making big changes, so we can now appreciate the impact of them. Also, we started noticing patterns that resulted from the change that show the need for further adjustments.

The metrics we’ve introduced (so far) are:

  • % people with job information: represents people in our system that have at least one job record
  • % people with a primary job: shows people in our system that have one of their jobs marked as primary
  • % records on ACT with correct job info: measures people with a correct job role on ACT
  • % records on ACT with correct organisation info: people that do have a job record but their organisation info is unknown and appears as such on ACT
  • number of people with unknown organisation info: counts all the people on People Data, not just those who have access to ACT (which are a subset of this)
  • number of people with unknown job titles: we have job information but we don’t know their job title
  • number of people with both an unknown job title and organisation info

We are also tracking the following metrics, but need to spend more time analysing the results

  • people without sponsor by job role
  • people without manager by job role
  • people without location information by job role

Examples of Job Roles are: Member of the House of Commons, Staff of the House of Lords, Staff of the Digital Service, etc.

So let’s see some graphs

Graph 1

Graph 1 shows the positive impact of enhancements carried out by David and manual fixes applied by Noel. 100% of people have job information associated with them. This has positive impacts in subscribing systems like ACT the Learning & Development system

Let’s see the impact to ACT below:

Graph 2

The enhancement implemented in late May 2016 is shown by the yellow line going up steeply. The percentage of correct records went up from 67% to 98%, a 31% increase, and we are now only 2% away from reaching our target.

Ok, but how will all this work benefit me, the parliamentary IT end user?

This is a fair question and I believe we must address this, as after all the geeky talk (hopefully not too jargon loaded), what our job comes down to is providing benefits for the people in our organisation; enhancing their “IT experience”.

There’s a series of benefits that we can mention:

Regarding ACT:

  • your own job information will be correct
  • the Learning & Development Team periodically runs reports to see who has and who hasn’t done specific training; they now have more reliable organisational information to track it by each team
  • Other changes that have been done as part of the integration enhancement for ACT are what we call “once on act, always on act”. To be on ACT you must have or have had a Parliamentary network account; but if you no longer have one and still work in Parliament, your information on ACT will continue to be updated. This didn’t happen before.

Regarding Cherwell: your information is now more complete -previously your job information or organisation information may not have been shown here, now it is

Regarding Active Directory [2]: we are currently working to turn on the PD to AD link again. This will be highly visible. So if

      • you changed your phone number (and notified Telecoms [3])
      • you changed jobs (and HR or the Pass Office updated your record)
      • Your manager changed (and HR updated your record) then your information will be automatically updated, making it visible on Skype for Business and your Outlook contact list.

So, in general terms, anywhere you can see your job information - that relies on people data - now has more accurate data, and hopefully will be even more accurate soon.

Thanks for reading.  Matias Noe.


[1] I strongly suggest you start trying to remember that nickname as I wouldn’t mind people calling me that

[2] AD (Active Directory) is the system where the Parliamentary Network accounts are managed.

[3] Something that you should do and we kindly encourage you to.

Sharing and comments

Share this page

1 comment

  1. Comment by Henry Lau posted on

    Great post!