Skip to content
Showing posts with label Data quality. Show all posts
Showing posts with label Data quality. Show all posts

Monday, July 13, 2026

Travel funnels from the other side: Naver, Google Maps and reviews paid in gifts in Korea

Part of Travel funnels from the other side, a series on what I learn as a traveller about the booking funnels I usually measure.

In June 2026 I travelled around South Korea. After years of measuring how customers search, book and pay, it was the second trip where I watched myself do it in a market that works differently. The first was Chile and Brazil in 2024. Korea was harder, and more interesting for anyone who works in travel.

Finding my way without Google

Google Maps shows places in Korea, but it would not give me walking or driving directions. The reason is a long-running dispute over taking detailed map data out of the country (background on Wikipedia). In February 2026 the government approved Google's request under strict conditions, but in June the directions still did not work for me. So I did what Koreans do: I moved around with Naver Map and KakaoMap, and I searched on Naver.

Within a couple of days most of my trip happened on Korean platforms. The searches I made, the places I saved and the routes I followed never touched the tools a Western travel brand usually measures.

Booking and paying as a foreigner

Two things stopped me again and again. Many websites asked for a Korean phone number to verify the account, and my foreign card was rejected on several of them. I had seen the same pattern in Chile: the customer is there, wants to pay, and the form decides they cannot.

A gift for a review

The thing that surprised me most happened in shops and restaurants. Several of them offered me a small gift if I left a review on Google or on Naver.

Google's rules for Maps reviews do not allow this. Its Maps content policy lists offering incentives in exchange for reviews as fake engagement. I do not know how strictly Naver applies its own rules, and I am not judging the businesses, which were following what looked like a common local practice. As an analyst, what I noticed was the effect on the data: a rating built partly on rewarded reviews means something different from a rating built on unprompted ones.

What a travel company could take from it

If I worked on the data side of a travel brand selling into Korea, I would look at three things.

  1. Search demand outside Google. A brand that measures and bids only on Google sees part of the market. Naver has its own search ads and its own webmaster tool, Naver Search Advisor, and the queries that matter may only show up there.
  2. Review data as a data quality question. If ratings feed a ranking, a recommendation or an analysis that compares markets, they need context: where incentives are common, a 4.8 is a different number. Sudden jumps in the number of reviews for one venue are worth flagging before anyone reads the average.
  3. The checkout for foreign customers. Errors by form field, split by the visitor's country, show quickly when a phone verification step or a card check is turning away people who wanted to buy.

A question to finish

When you compare ratings or search demand across countries, do you know which platform each market really uses, and how its reviews were collected?

Embudos de viaje desde el otro lado: Naver, Google Maps y reseñas a cambio de regalos en Corea

Forma parte de Embudos de viaje desde el otro lado, una serie sobre lo que aprendo como viajera de los embudos de reserva que suelo medir.

En junio de 2026 viajé por Corea del Sur. Después de años midiendo cómo buscan, reservan y pagan los clientes, era el segundo viaje en el que me veía hacerlo en un mercado que funciona de otra manera. El primero fue Chile y Brasil, en 2024. Corea fue más difícil, y más interesante para quien trabaja en viajes.

Orientarse sin Google

Google Maps muestra los sitios en Corea, pero no me daba indicaciones para ir a pie ni en coche. El motivo es un conflicto que viene de lejos sobre sacar del país datos cartográficos detallados (contexto en Wikipedia). En febrero de 2026 el Gobierno aprobó la petición de Google con condiciones estrictas, pero en junio las indicaciones seguían sin funcionarme. Así que hice lo que hacen los coreanos: me moví con Naver Map y KakaoMap, y busqué en Naver.

En un par de días, casi todo mi viaje pasaba por plataformas coreanas. Las búsquedas que hacía, los sitios que guardaba y las rutas que seguía nunca pasaban por las herramientas que suele medir una marca de viajes occidental.

Reservar y pagar como extranjera

Dos cosas me frenaban una y otra vez. Muchas webs pedían un número de teléfono coreano para verificar la cuenta, y mi tarjeta extranjera fue rechazada en varias. Ya lo había visto en Chile: la clienta está ahí, quiere pagar, y el formulario decide que no puede.

Un regalo por una reseña

Lo que más me sorprendió pasó en tiendas y restaurantes. En varios me ofrecieron un pequeño regalo si dejaba una reseña en Google o en Naver.

Las normas de Google para las reseñas de Maps no lo permiten. Su política de contenido de Maps considera interacción falsa ofrecer incentivos a cambio de reseñas. No sé con qué rigor aplica Naver sus propias normas, y no juzgo a los negocios, que seguían lo que parecía una práctica local habitual. Como analista, lo que me llamó la atención fue el efecto en el dato: una valoración construida en parte con reseñas premiadas significa algo distinto de una construida con reseñas espontáneas.

Qué podría sacar de aquí una empresa de viajes

Si trabajara en datos en una marca de viajes que vende en Corea, miraría tres cosas.

  1. La demanda de búsqueda fuera de Google. Una marca que solo mide y puja en Google ve una parte del mercado. Naver tiene sus propios anuncios de búsqueda y su propia herramienta para webmasters, Naver Search Advisor, y las búsquedas que importan pueden aparecer solo ahí.
  2. Los datos de reseñas como un problema de calidad del dato. Si las valoraciones alimentan un ranking, una recomendación o un análisis que compara mercados, necesitan contexto: donde los incentivos son habituales, un 4,8 es otro número. Los saltos bruscos en el número de reseñas de un local merecen una alerta antes de que nadie lea la media.
  3. El checkout para clientes extranjeros. Los errores por campo del formulario, separados por el país del visitante, muestran enseguida cuándo una verificación por teléfono o una comprobación de tarjeta está echando a gente que quería comprar.

Una pregunta para terminar

Cuando comparas valoraciones o demanda de búsqueda entre países, ¿sabes qué plataforma usa de verdad cada mercado y cómo se recogieron sus reseñas?

Monday, June 15, 2026

Writing analytics specs that developers can build

In UK government digital teams I wrote the specifications for my own team's analytics work, as user stories with acceptance criteria. The rule I took from it is easy to state: an analytics request is ready for development when someone other than the analyst can test it.

The situation

I worked as an analyst in UK government digital teams, at GDS and the Department for Education, often in several teams at the same time. The teams were multidisciplinary and worked in Agile, with product and delivery managers, developers, designers and user researchers on the same service.

Part of the work was digital identity. In the UK that is harder than in many countries, because there is no national identity card to check people against, so a service has to build confidence in who someone is from several weaker pieces of evidence. Government is still working on it: in February 2026 HMRC started signing up new customers through GOV.UK One Login, which GOV.UK said was then available for over 200 government services.

In a team like that, an analytics request competes with everything else on the board. If it arrives as "can we track the identity check?", the developer has to guess what to send, when and with which values. The guesses come back at user acceptance testing (UAT), when changing them is expensive.

What I did

For my own team, I wrote each analytics task as a user story with acceptance criteria. The story said who needed the data and which decision it supported. The criteria said what the developer had to build and how we would check it in UAT.

The example below is invented for this post and has no detail from a real service, but it has the shape I used.

As a performance analyst, I want an event when a user leaves the identity check step, so that we can report completion by step and see where people drop out.

  • When the user leaves the step, one event named identity_check_step_end is sent.
  • The event has an outcome parameter with one of these values: passed, failed, exited.
  • The event sends no name, date of birth, document number or free text.
  • In UAT, one attempt with each outcome produces exactly one event per attempt in the test property.

The privacy line sat in the criteria on purpose. In an identity service the safest personal data is the data the event never carries, and a written criterion can be tested in UAT like any other.

Writing the check first also moves the questions earlier. Edge cases come up while the story is being refined, when an answer is a comment on the ticket.

The team also automated its recurring reporting with Python and ETL pipelines. That cut the analytics effort on reports by around three quarters, and weekly reports went out on time to more than 30 stakeholders. It was a team achievement, built in Agile with everyone on the team.

What I learned

Acceptance criteria that cannot fail are a wish list. Each line has to describe something a tester can see happen or not happen. The analyst is the best person to write those lines, because the analyst knows what the data will be used for and which mistakes would make it useless.

What I would do today

The GOV.UK Service Manual already asks teams to have a performance analyst from the start of discovery and to work out their measurements while the service is built (Service Manual). I would keep that and add four things.

  1. Link the tracking plan to the stories, so each event points to the story that created it and to the metric it feeds.
  2. Check event names and allowed values automatically in the test environment, so a typo fails before release.
  3. Agree once with the privacy team a list of fields that never go into analytics, and paste it into every story.
  4. Write acceptance criteria for AI features in the same way. A feature built on a large language model also needs written cases it must pass before release, and someone other than its builder should be able to run them.


Monday, April 20, 2026

Agree the metric before you build the dashboard

At JD Sports Group, marketing and ecommerce each had their own figures and neither trusted the other's. The programme that fixed it started with definitions agreed by the managers, and the dashboards came afterwards.

The situation

From 2021 to 2024 I was Senior Digital & Marketing Data Analyst at JD Sports Group and product owner of a programme of 8 projects to improve measurement end to end. When the programme started, each manager had their own objectives and their own KPIs. Marketing did not trust the ecommerce figures, ecommerce did not trust the marketing ones, and meetings that should have ended in a decision ended in a discussion about whose report was right.

The request that usually arrives in that situation is a new dashboard. Built on top of the disagreement, it would have shown both sets of numbers on one screen, and the discussion would have carried on in front of it.

What I did

I started with a measurement plan, before choosing any tool. I ran sessions with the managers to agree the business objectives first, and then the KPIs that would show whether we were meeting them, for internal users and for customers.

To give it a structure I used the GDS Performance Framework, which separates primary and secondary metrics. Each area ended up with a small number of metrics that mattered, each with one shared definition behind it. GOV.UK has since withdrawn that framework page and points readers to its Service Manual guide on measuring success, which asks teams to work out their measurements while they build the service.

I split the roadmap in two. The strategic part was objectives, metric definitions and ownership. The operational part was instrumenting the tracking with developers and building dashboards. We ran it as an agile programme, with a Trello board for the programme and Jira for the delivery work with developers. New requests arrived every week, so the backlog kept moving, and I prioritised by dependency and by how many teams a piece of work would unblock. A metric was defined before it was implemented.

Dashboards were released one implementation at a time. After each one, people could see an operational dashboard, use it and tell us what was missing, and most of the refinement of the metrics happened at that stage. When a new implementation moved into business as usual, a short survey checked whether it met the need it had been built for.

The programme delivered its 8 projects with 90% stakeholder satisfaction, and adoption of analytics grew 70% year on year. Adoption is the figure I value most, because it shows that people started coming to the data to settle questions.

What I learned

Implementing a metric that nobody has agreed on moves the argument downstream. In a meeting the disagreement is visible and cheap to resolve. Inside a dashboard it looks like a data quality problem, and tracing it back to a definition takes much longer.

The second lesson came from releasing early. A definition agreed in a workshop is still a draft until people use it with real data, and the first dashboard was the cheapest place to find out what the draft had missed.

What I would do today

Creating a metric keeps getting easier. With Tag Manager templates and point-and-click tagging, a new event can be live in minutes. When an event takes minutes to create, the definition is the part of the work that needs an owner.

If I ran the same programme now, I would add four habits.

  1. A written definition for every primary metric, with the business question it answers, the formula, the data source, the filters, the owner and the date it was agreed.
  2. A change log that the dashboard shows. When a definition changes, the date appears on the chart, so nobody reads a step in the line as a change in the business.
  3. No new event in the tracking plan unless it feeds a metric that has a definition.
  4. A review of the secondary metrics every quarter, to retire the ones nobody opens.

Before your next dashboard

Take the headline metric of the dashboard you are about to build and ask two managers who use it to write down, separately, how it is calculated. If the answers differ, that conversation comes first, and the dashboard can wait a week.

Monday, February 16, 2026

A hole in the checkout funnel: fix the tracking before you redesign the page

At JD Sports Group the business was ready to redesign the checkout because the funnel looked broken. The first problem I found was a page nobody had tracked, and fixing it changed what the rest of the project could do.

The situation

I was Senior Digital & Marketing Data Analyst at JD Sports Group from 2021 to 2024. One project started with a clear request: the checkout funnel is broken and needs redesigning. Several teams had a view on what was wrong and what to change, and the pressure was to start changing things.

What I did

I started with the analysis. Compared with benchmark, the funnel numbers were off in a way that did not fit a checkout that had not changed recently. Following the drop led me to a page that had been added to the checkout at some point and never tracked. Customers were passing through a step the funnel could not see, which left a hole in the report.

We fixed the tracking first. That gave the teams a funnel they could trust, and it gave me the credibility to ask for more advanced tracking, which was approved.

The numbers then showed where people left the checkout, but they could not say why. I added a survey with Hotjar, the tool the team already had, and used the OpenAI API to clean and categorise the open-text answers. Payment problems came out as the main theme, backed by what customers had written.

Session recordings came next, but the volume was too high to turn into a priority list. So I changed approach and worked with Customer Care to put live chat on the checkout during working hours, on one condition: agents logged the reason for each conversation in Salesforce. That gave us the reasons directly from customers, every week.

There were competing requests. Marketing wanted Customer Care to run video calls with customers, and Customer Care wanted the chat on product pages. I argued for the checkout first by walking both teams through the funnel step by step, showing what data we already had at each stage and what each option would tell us that we did not know yet. The reporting stayed simple, with charts and a weekly report, and the teams started reusing it in their own meetings. Once the report was useful to them, the priority was no longer up for debate.

The result, and how it was measured

Checkout conversion improved by 4% (relative) after the changes. That figure comes from a before-and-after comparison against the baseline before launch. There was no A/B test, so the comparison cannot separate the effect of the changes from seasonality or from anything else that moved in the same period. When I present the number, that limit goes next to it.

The weekly report outlived the project, and Customer Care was still using it in their own meetings long after it closed. It also showed me the weak point. As Customer Care hired new people, the logging discipline dropped and the reason data became less reliable, so the reports kept coming out while their value went down. Retraining became a recurring task, built into how the report was maintained.

What I learned

A funnel is a data product, and the cheapest data quality control is checking that every step in it is measured before anyone draws conclusions from it. The epilogue taught me the second lesson: data that depends on people typing a reason into a CRM needs the same upkeep as a pipeline, because adoption work does not end at handover.

What I would do today

I would add four things.

  • Tracking in the definition of done. A new checkout step does not go live until its event is in the tracking plan and a test confirms that the event arrives.
  • An automated completeness check on the funnel. Pages inside the checkout path that receive sessions and send no funnel event should raise an alert.
  • The LLM categorisation treated as a measurement instrument. Personal data comes out of the free text before it goes to an external API, the prompt and model version are recorded, and a sample of answers coded by a person checks that the categories still hold.
  • The comparison agreed before launch: the baseline window, the change being measured and the list of other releases in the same period. If the traffic allows it, a holdout group as well.

Try this on your own funnel

Pick your highest-value funnel and list every page a customer can see between the basket and the confirmation page. Then check that each one sends an event. If the two lists do not match, that is the first finding to take to the redesign meeting.