This is the Data & Search team's fourth set of fortnight notes. The idea is to keep an ongoing, public record of what we're doing. We'll iterate the notes with a view to making them better over time.
We'd appreciate any feedback to @dasbarrett, barrettd@parliament.uk, or via the comments.
Here's what's been happening over the past 2 weeks:
On 21 September Dan and Kheira had a conference call with Richard and Tom from the ODI and agreed final steps for publication of the report on Parliament's data we commissioned. It's coming soon.
At our 'data day' (planning day) on 28 September the whole team were there for the first time. It worked well, we will repeat it once a month. We cover several different areas of work as a team, so an overarching theme was how we can avoid working in silos, collaborate and cross skill.
On 29 September Aidan, Michael, Robert and Samu visited the Web Science Institute at the University of Southampton. They spent the day with an eclectic mix of academics discussing the work of Data and Search team and where the two parties might be able to work together. Aidan is writing a blog post about it.
Aidan has made further progress on contracts, procurement and financial planning. Aidan has also made progress on getting some new kit for the team.
Samu had a session with Aidan to explain the proposed design of our virtual infrastructure.
Michael had a further domain modelling session with the Commons Public Bill Office.
Michael and Ganesh have continued to work on Member data. Samu, Wojciech and Mike have been doing the same. Both efforts are combined and for the new website.
Michael finalised the brief for a workshop about Parliament's data at Newspeak House on 20 October with Ed Sapeira and the publicity went out.
Michael coordinated a conversation between Professor Michael Rush and the Wikimedia Foundation around the use of the Professor's historical Member data.
In between holiday, Robert worked on search prototype goals with Aidan, and had an accelerated introduction to Microsoft Azure with Samu and Wojciech. The result was a prototype environment ready to use to iterate the prototype.
Robert had a recruitment product team day with Julie. The result for this team is a job description for a Data Analyst that has been sent for approval.
Mat, David and Noel have had one new system integration go live, with another in test. Mat has also been reviewing two future integrations in collaboration with business colleagues. Data from our online training system has been reviewed and cleansed by the team, and testing for the Events Management System is underway.
At our show and tell on 4 October Mat presented on progress made with improvements to data from some of our internal systems, with proof from metrics that the team gather.
Mat starting working with Samu on prototypes that use Logic Apps - early steps towards upgrading our tools.
Wojciech had a successful first attempt on orchestration for Member core attributes and constituencies for the data platform. Data from relevant sources is populating the platform on a schedule.
Mike's work on maintenance automation for data.parliament.uk resulted in an increase in uptime as expected. However, we did have a prolonged outage on the weekend of 1 October, so further work is required.
Dan spent time with Steve (our senior business sponsor) to update on progress in advance of the afternoon we'll spend with him on 12 October. Dan is going to be working on a road map for the data services as a result.
Dan had discussions with Jamie and Matt about the coordinated plan of work between business systems, data and website in order to build the new website. Looking forward to the website road map as a result.
Robert gave Dan an introduction to GitHub.
The search on Historic Hansard went down unexpectedly but is now back up thanks to our colleague Joe and the Knowledge Applications Team. Robert and Dan did a fair amount of customer service and support as a result of this, the data.parliament outage and some other requests via Twitter.