Sprint 4 (of 7) notes

Tweaking elastic, building sentences and show & tells have been the order of this sprint – as it probably has been on a playgroup somewhere as well – though that’s where the similarity with my project stops ๐Ÿ˜‰

Unpicking elastic search

The initial implementation saw us using match query which literally searches any of the words entered in the search box, exactly as typed. It’s configured to search titles, summary and description (weighted in that order) of each opportunity and while the results that came back were ‘right’, it wasn’t immediately clear to the user why they were right (in instances where the match is in the depths of the opportunity or generic words such as ‘management’ in ‘waste water management’ would therefore return all sorts.

This sprint, we’ve worked up two instances for usability testing side by side: fuzzy (to allow for typos) and phrase matching (to consider the proximity of words to each other).


A few of the team are of to Venturefest Manchester and New Scientist Live for some pop up testingย  for us to gauge which, if any, instances bring back more useful results.


Building sentences

Last sprint highlighted the confusion users faced when starting a search by seeing the full list to then refine.

Screen Shot 2016-09-19 at 8.24.25 PM.png

I suppose it’s not unlike walking into a supermarket which shows everything on offer, and you walk around to pick what you want. Everyone’s used to that, products are consistently laid out; fruit and veg at the front, bakery somewhere near the back etc. But online search is a hugely different kettle of fish.

We’re trying a concept of building a sentence which is akin to how an advisor would help a client search the database if face to face.

We’ve got two routes: one which is more form-like and this one which is more like building a sentence.

sentence builder.gif


We’ll test both this coming sprint, but alternating which one tests first, to avoid skewing the result. While the team is in Manchester, I’ll nip off to Bolton to meet a user who has assisted digital needs to see how she gets on with them. Before we get too excited with our creation we need to work up whichever one performs best at testing for all our audiences with their varying needs.

Show and tells

Each sprint retrospective, I record and 10-ish minute screen/audio canter through what the team has done, and send it to stakeholders for their information and comment. The format is working well and this last sprint I had a healthy slot at the all staff webinar.

The objective was to inform a wider group of stakeholders (โœ“) kick the tyres on a few things and test the reaction (โœ“) and to run a couple of web environment / Salesforce / GovDelivery hack ideas I had by then to see if there’s sufficient advisor (user) need to add them to the backlog (โœ“).

Those hack ideas went down well…

Make the website the end point for the user after a first meet with an advisor.

  • Scenario: Often after a first advisor/client meet, an advisor would follow up with an email pointing to a few relevant opportunities for the client. Advisor colleagues won’t know the detail nor will anyone know opens or clicks.
  • Idea:
    1. The advisor searches the database in the backend to select a couple of things.
    2. An email button opens up the things which the advisor can write a message
    3. Click send which transports through GovDelivery which therefore adds the email to the contact on Salesforce as well as any engagement stats.

Segmenting the audience by keywords we know they’re interested in

  • Scenario: Advisor is running an event about the circular economy, so wants to invite clients in that field
  • Idea:
    1. Again searching the backend for relevant keywords – stuff like ‘recyclable, environment, waste water – brings up all the clients subscribed to those words and clients sent opportunities with those words in, perhaps from different keywords.
    2. Advisor can select users to add to a GovDelivery topic which then
    3. is visible in GovDelivery where the email template is created
    4. Engagement stats are then passed back to each Salesforce contact.

This would be even cooler if the clients results showed number of clicks for the searched subject, should we need to be selective on which clients we sent to. But, first thing first. There’s a huge organisational task of cleansing the data we have: currently 1/3 of our clients receiving these alerts do so with optimised the keywords, the rest receive everything which is a legacy thing on our side.

It’ll pay dividends though. Conversions from that 1/3 is twice as effective.


That’s sprint 3 done

Another fortnight passes, a lot’s going on, but this week I had the (only slight) realisation of the mountain still to climb. Doable though. Very doable.

User research recruitment

Feels as though this is an ongoing challenge for agile teams in government. An ongoing tussle between recruitment suppliers understanding and delivering the brief and participants participating can make things a little unpredictable when relying on (a different sample of) user input at each sprint to inform what you’re making.

I’ve taken matters into my own hands by recruiting myself, which remains to be seen if it’s the right thing to do. I’m using Twitter to find ‘innovative small businesses’ by scraping large accounts like @startupstowers, @beisgovuk and @britishchambers and building lists with auidiense of accounts with certain keywords in the bio. I’m then using Dr Tweety to export that info and manually(!) flicking through to get a shortlist to politely and respectfully approach.

The idea is to get a mix of people who know about the Innovate UK family (@innovateuk, @EEN_UK, @KTNUK and @Catapult_UK – and all 50 sub twitter accounts) and those who don’t. And also aim for a split across sectors. All in the name of a balanced view, which may also throw up some contrasting insight.

We’re on a shortlist of 60 names and have visualised the relationships between them in gephi, which is helpful to see which of the segments described above are under represented.

network no labels.png

I’m keen to see if a visualisation like this could be used for showing the rank of users needs and how (if they do) vary across the segments. We’ll see if patterns start to emerge. One of the reasons behind my thinking for this is that too often research is done then left collecting dust. I like the idea of something this project (and others) can build on over time and interrogate.

Usability testing

Sticking with the user, we ran our first round of showing some folk our work last sprint.

We’re starting from a GOV.UK look and feel, albeit different branding, and will deviate away at points where users got a little stuck. That’s the joy of a GOV.UK exemption ๐Ÿ™‚

A big thing we’re taking to sprint 4 making the search easier for users to get their head around.

Search Results (GDS) - Selected

While showing all results is the default starting point on GOV.UK, where you tweak the parameters to refine the results, users were overwhelmed so we need to explore the options.

There are tweaks to microcopy, highlighting keywords in results as well as tagging on the cards, as is some sort of sentence builder for the user to play with first, which then returns the results page.

A big challenge we’ve got with the API’d content is the occasional contradictory headlines and descriptions where users don’t feel they’re looking at relevant results (from the headlines alone), but in reading the description, they realise it is relevant.

Wider team

I also kicked off a series of user stories that the wider team of Methods Digital (who are configuring Salesforce) and GovDelivery (current technology in the frame to send the email / sms messages) are feeding into and divvying up who’s doing what. No doubt loads of work there to nail dataflows, privacy etc.