The goings-on in the Data Search world...13/02/2017

The Data and Search team have made good progress over the past two weeks. In this eleventh set of fortnight notes I will try and encapsulate all those happenings!

Ontology

This was the subject of our show and tell on 7th February. Aligning our physical ontology to the conceptual one is now complete. You can see the work in progress here. We had to do this because the model that we started using for our development of the data service got out of sync with Michael’s modelling work, and so we ended up with two slightly different ontologies. These two now match, so now work can start on modifying our data accordingly.

Modifying our orchestration to match the physical ontology is also complete, so our instance data now conforms to the domain model.

New Data Platform

On 19/01 there was a session with colleagues from Business Systems about a release to one their applications that requires work on data.parliament.uk. The session was productive where the data team had an opportunity to share information about the current and new data platform. Samu and his team implemented the change required.

Samu, Michael and Aidan agreed a statement of work for with Southampton University covering technical implementation of the data platform. Work on contracts is in progress

Accreditation

Samu and Aidan have been working hard on security accreditation. Aidan met with colleagues in the Infrastructure Programme to discuss next steps for the Data and Search platform’s hosting and infrastructure in Microsoft Azure. They also met with colleagues in Cyber Security to discuss the accreditation process and what is still required from Data and Search. Aidan has been working to procure some expert consultancy to work with the team on making our environments secure.

Search

Jamie, Dan, Samu, Robert and Dia had a good meeting where the technical approach and solution for a first step to improve search on parliament.uk was agreed. Dia, Henry and Stephen met to discuss the user research findings for search. Aidan engaged with suppliers to procure third party services that will be used to develop the new search service for Parliament

Corporate Management Information Dashboards

Dan has made some good progress on this work, drawing together a new team to scope and deliver it. Matias and Samu have started researching and experimenting with Microsoft’s PowerBI tools. In the last two weeks as part of prototyping they were able to install an on-premises data gateway, configure a local database as a data-source, author a report based on this data and share a dashboard that features it. Dia has a first draft spreadsheet of our understanding of demand. Mat and Laura made progress on security accreditation. Mat worked with users to develop a minimum data set for the first dashboard the team have been tasked with, which is due on 16 March.

Product Roadmap

Dia shared the first draft of the product roadmap. It’s a work in progress document but is taking shape

Support

The team resolved two difficult issues with systems that they support but didn’t build in the past two weeks.

After developing and implementing a fix, work is ongoing to re-process a build-up of Hansard material that was causing broken links to contributions on our internal search.

An issue with search on our intranet was also resolved, where a variety of pages weren’t appearing in the results. This was challenging to work through because documentation was poor, but in the end it was successful. Kunal, Mike, Samu and Wojciech got through these issues with support from colleagues Joe and Sarah.

Other Interesting Stuff

Dia and Michael had a conference call with the Welsh Assembly to share experience on data, search and making progress on the new website. They also discussed how we work in the digital service.

Michael had a good meeting with Anya and petitions committee clerks to review our flowchart as part of work to model petitions data.

Michael did a version of his modelling parliaments blog post for the Political Studies Association with Ben Worthy .

Matt from the website team showed us how Pugin (the new Pattern Library) works and how it fits into the overall website architecture. There is some great software design there. It was good to see how data from our service is becoming a website.

Samu attended a talk by Microsoft about widespread adoption of 'DevOps' culture and iterative development practices in their engineering teams. Hearing about Microsoft’s transformation story was fascinating.

The team have been working on analysis of our hosting spend at a higher level of detail than we’ve done previously. This will help us meaningfully forecast to cost of our infrastructure. Raphael will start working with this data in PowerBI, which will help create reports and dashboards for technical monitoring and executive oversight.

Aidan, with significant input from Emma and Dan, submitted the application for the Digital Service, and in particular Data & Search, to be a host organisation for the Ford-Mozilla Open Web Fellow Program. We should hear if we're selected before the end of February."

Samu, Jamie, Robert, Mat, Dan, Aidan, Henry, Dia and Chris had a first attempt at recording a team podcast.

Michael went to the #NotWestminster event in Huddersfield on 10 February

And finally a plug for our Parliament, Data and Democracy Meetup on 21 February 7pm at Newspeak House in Shoreditch.

We'd appreciate any feedback to @DN78, nagd@parliament.uk, or via the comments.

Blog

Parliamentary Digital Service