AI literacy under Article 4 is an adoption problem

Since July 2026 the AI Act asks providers and deployers to take measures that support their staff's AI literacy, without requiring any specific level from any individual. I learned how that kind of duty plays out from analytics adoption at JD Sports Group, where getting people trained was the easy part and keeping the habit was the real work.

What changed in Article 4

Article 4 has applied since 2 February 2025. The original text asked providers and deployers to take measures to ensure, to their best extent, a sufficient level of AI literacy among their staff. The Digital Omnibus on AI (Regulation (EU) 2026/1744, in force since 27 July 2026) rewrote it. Providers and deployers now have to take measures to support the development of AI literacy of their staff, the text adds that this does not require any specific level of AI literacy of any individual, and the Commission has to publish practical examples of how to comply (AI Act Explorer).

The Commission's AI literacy Q&A says there is no need for a certificate, that relying on the instructions for use is not sufficient, and that organisations can keep an internal record of their training and guidance.

Read quickly, the new wording sounds lighter. In practice it leaves each organisation to decide what support means for its people and how it will know whether the support is working. I had to answer the same question for analytics.

What analytics adoption taught me

At JD Sports Group I was product owner of a programme of 8 measurement projects. The result I value most from it is a measure of behaviour: adoption of analytics grew 70% year on year. Stakeholder satisfaction was 90%, which was good to hear, but adoption told us whether people used the data when they made decisions.

Two practices were part of the programme from the start. Each implementation came with an operational dashboard, so people learned on the tool they would use. When an implementation moved into business as usual, a short survey checked whether it met the need it was built for.

The checkout project taught me the other half. Customer Care logged the reason for each live chat in Salesforce, and the weekly report built on those reasons was still used in their meetings long after the project closed. As the team hired new people, the logging discipline dropped and the reasons became less reliable, while the report kept arriving every week. The plan had been a one-off handover, so I made periodic retraining of Customer Care part of how the report was maintained.

Why AI tools behave the same way

An AI tool launched with one training session is in the same position as that report. People who join after the session never had it, the tool changes with each model update, and the habits that make output safe to use fade quietly: checking a claim against its source, or knowing which customer data must never go into a prompt. Nothing breaks on the day the habit goes, which is why a course completion rate says little about it.

Day to day I use AI to automate repetitive tasks, and my own time goes on verifying the output before I sign it off. That verification step is the first thing I would teach.

What I would do today

1. Start with an inventory of who uses which AI tools and for what. The role decides what literacy means: someone drafting customer replies with an assistant needs different knowledge from an analyst building a feature on a large language model.
2. Put AI guidance into onboarding, and repeat it on a date in the calendar and whenever a tool or its use changes.
3. Measure what people do with the tools. Two useful signals are the share of AI-assisted outputs that someone reviewed before they went out, and the incidents and near misses that people report.
4. Keep a dated record of the support each group received. The Q&A asks for no certificate, and a record is the simplest way to show what was done and when.

Travel funnels from the other side: Naver, Google Maps and reviews paid in gifts in Korea

Part of Travel funnels from the other side, a series on what I learn as a traveller about the booking funnels I usually measure.

In June 2026 I travelled around South Korea. After years of measuring how customers search, book and pay, it was the second trip where I watched myself do it in a market that works differently. The first was Chile and Brazil in 2024. Korea was harder, and more interesting for anyone who works in travel.

Finding my way without Google

Google Maps shows places in Korea, but it would not give me walking or driving directions. The reason is a long-running dispute over taking detailed map data out of the country (background on Wikipedia). In February 2026 the government approved Google's request under strict conditions, but in June the directions still did not work for me. So I did what Koreans do: I moved around with Naver Map and KakaoMap, and I searched on Naver.

Within a couple of days most of my trip happened on Korean platforms. The searches I made, the places I saved and the routes I followed never touched the tools a Western travel brand usually measures.

Booking and paying as a foreigner

Two things stopped me again and again. Many websites asked for a Korean phone number to verify the account, and my foreign card was rejected on several of them. I had seen the same pattern in Chile: the customer is there, wants to pay, and the form decides they cannot.

A gift for a review

The thing that surprised me most happened in shops and restaurants. Several of them offered me a small gift if I left a review on Google or on Naver.

Google's rules for Maps reviews do not allow this. Its Maps content policy lists offering incentives in exchange for reviews as fake engagement. I do not know how strictly Naver applies its own rules, and I am not judging the businesses, which were following what looked like a common local practice. As an analyst, what I noticed was the effect on the data: a rating built partly on rewarded reviews means something different from a rating built on unprompted ones.

What a travel company could take from it

If I worked on the data side of a travel brand selling into Korea, I would look at three things.

1. Search demand outside Google. A brand that measures and bids only on Google sees part of the market. Naver has its own search ads and its own webmaster tool, Naver Search Advisor, and the queries that matter may only show up there.
2. Review data as a data quality question. If ratings feed a ranking, a recommendation or an analysis that compares markets, they need context: where incentives are common, a 4.8 is a different number. Sudden jumps in the number of reviews for one venue are worth flagging before anyone reads the average.
3. The checkout for foreign customers. Errors by form field, split by the visitor's country, show quickly when a phone verification step or a card check is turning away people who wanted to buy.

A question to finish

When you compare ratings or search demand across countries, do you know which platform each market really uses, and how its reviews were collected?

Embudos de viaje desde el otro lado: Naver, Google Maps y reseñas a cambio de regalos en Corea

Forma parte de Embudos de viaje desde el otro lado, una serie sobre lo que aprendo como viajera de los embudos de reserva que suelo medir.

En junio de 2026 viajé por Corea del Sur. Después de años midiendo cómo buscan, reservan y pagan los clientes, era el segundo viaje en el que me veía hacerlo en un mercado que funciona de otra manera. El primero fue Chile y Brasil, en 2024. Corea fue más difícil, y más interesante para quien trabaja en viajes.

Orientarse sin Google

Google Maps muestra los sitios en Corea, pero no me daba indicaciones para ir a pie ni en coche. El motivo es un conflicto que viene de lejos sobre sacar del país datos cartográficos detallados (contexto en Wikipedia). En febrero de 2026 el Gobierno aprobó la petición de Google con condiciones estrictas, pero en junio las indicaciones seguían sin funcionarme. Así que hice lo que hacen los coreanos: me moví con Naver Map y KakaoMap, y busqué en Naver.

En un par de días, casi todo mi viaje pasaba por plataformas coreanas. Las búsquedas que hacía, los sitios que guardaba y las rutas que seguía nunca pasaban por las herramientas que suele medir una marca de viajes occidental.

Reservar y pagar como extranjera

Dos cosas me frenaban una y otra vez. Muchas webs pedían un número de teléfono coreano para verificar la cuenta, y mi tarjeta extranjera fue rechazada en varias. Ya lo había visto en Chile: la clienta está ahí, quiere pagar, y el formulario decide que no puede.

Un regalo por una reseña

Lo que más me sorprendió pasó en tiendas y restaurantes. En varios me ofrecieron un pequeño regalo si dejaba una reseña en Google o en Naver.

Las normas de Google para las reseñas de Maps no lo permiten. Su política de contenido de Maps considera interacción falsa ofrecer incentivos a cambio de reseñas. No sé con qué rigor aplica Naver sus propias normas, y no juzgo a los negocios, que seguían lo que parecía una práctica local habitual. Como analista, lo que me llamó la atención fue el efecto en el dato: una valoración construida en parte con reseñas premiadas significa algo distinto de una construida con reseñas espontáneas.

Qué podría sacar de aquí una empresa de viajes

Si trabajara en datos en una marca de viajes que vende en Corea, miraría tres cosas.

1. La demanda de búsqueda fuera de Google. Una marca que solo mide y puja en Google ve una parte del mercado. Naver tiene sus propios anuncios de búsqueda y su propia herramienta para webmasters, Naver Search Advisor, y las búsquedas que importan pueden aparecer solo ahí.
2. Los datos de reseñas como un problema de calidad del dato. Si las valoraciones alimentan un ranking, una recomendación o un análisis que compara mercados, necesitan contexto: donde los incentivos son habituales, un 4,8 es otro número. Los saltos bruscos en el número de reseñas de un local merecen una alerta antes de que nadie lea la media.
3. El checkout para clientes extranjeros. Los errores por campo del formulario, separados por el país del visitante, muestran enseguida cuándo una verificación por teléfono o una comprobación de tarjeta está echando a gente que quería comprar.

Una pregunta para terminar

Cuando comparas valoraciones o demanda de búsqueda entre países, ¿sabes qué plataforma usa de verdad cada mercado y cómo se recogieron sus reseñas?

Writing analytics specs that developers can build

In UK government digital teams I wrote the specifications for my own team's analytics work, as user stories with acceptance criteria. The rule I took from it is easy to state: an analytics request is ready for development when someone other than the analyst can test it.

The situation

I worked as an analyst in UK government digital teams, at GDS and the Department for Education, often in several teams at the same time. The teams were multidisciplinary and worked in Agile, with product and delivery managers, developers, designers and user researchers on the same service.

Part of the work was digital identity. In the UK that is harder than in many countries, because there is no national identity card to check people against, so a service has to build confidence in who someone is from several weaker pieces of evidence. Government is still working on it: in February 2026 HMRC started signing up new customers through GOV.UK One Login, which GOV.UK said was then available for over 200 government services.

In a team like that, an analytics request competes with everything else on the board. If it arrives as "can we track the identity check?", the developer has to guess what to send, when and with which values. The guesses come back at user acceptance testing (UAT), when changing them is expensive.

What I did

For my own team, I wrote each analytics task as a user story with acceptance criteria. The story said who needed the data and which decision it supported. The criteria said what the developer had to build and how we would check it in UAT.

The example below is invented for this post and has no detail from a real service, but it has the shape I used.

As a performance analyst, I want an event when a user leaves the identity check step, so that we can report completion by step and see where people drop out.

• When the user leaves the step, one event named identity_check_step_end is sent.
• The event has an outcome parameter with one of these values: passed, failed, exited.
• The event sends no name, date of birth, document number or free text.
• In UAT, one attempt with each outcome produces exactly one event per attempt in the test property.

The privacy line sat in the criteria on purpose. In an identity service the safest personal data is the data the event never carries, and a written criterion can be tested in UAT like any other.

Writing the check first also moves the questions earlier. Edge cases come up while the story is being refined, when an answer is a comment on the ticket.

The team also automated its recurring reporting with Python and ETL pipelines. That cut the analytics effort on reports by around three quarters, and weekly reports went out on time to more than 30 stakeholders. It was a team achievement, built in Agile with everyone on the team.

What I learned

Acceptance criteria that cannot fail are a wish list. Each line has to describe something a tester can see happen or not happen. The analyst is the best person to write those lines, because the analyst knows what the data will be used for and which mistakes would make it useless.

What I would do today

The GOV.UK Service Manual already asks teams to have a performance analyst from the start of discovery and to work out their measurements while the service is built (Service Manual). I would keep that and add four things.

1. Link the tracking plan to the stories, so each event points to the story that created it and to the metric it feeds.
2. Check event names and allowed values automatically in the test environment, so a typo fails before release.
3. Agree once with the privacy team a list of fields that never go into analytics, and paste it into every story.
4. Write acceptance criteria for AI features in the same way. A feature built on a large language model also needs written cases it must pass before release, and someone other than its builder should be able to run them.

Agree the metric before you build the dashboard

At JD Sports Group, marketing and ecommerce each had their own figures and neither trusted the other's. The programme that fixed it started with definitions agreed by the managers, and the dashboards came afterwards.

The situation

From 2021 to 2024 I was Senior Digital & Marketing Data Analyst at JD Sports Group and product owner of a programme of 8 projects to improve measurement end to end. When the programme started, each manager had their own objectives and their own KPIs. Marketing did not trust the ecommerce figures, ecommerce did not trust the marketing ones, and meetings that should have ended in a decision ended in a discussion about whose report was right.

The request that usually arrives in that situation is a new dashboard. Built on top of the disagreement, it would have shown both sets of numbers on one screen, and the discussion would have carried on in front of it.

What I did

I started with a measurement plan, before choosing any tool. I ran sessions with the managers to agree the business objectives first, and then the KPIs that would show whether we were meeting them, for internal users and for customers.

To give it a structure I used the GDS Performance Framework, which separates primary and secondary metrics. Each area ended up with a small number of metrics that mattered, each with one shared definition behind it. GOV.UK has since withdrawn that framework page and points readers to its Service Manual guide on measuring success, which asks teams to work out their measurements while they build the service.

I split the roadmap in two. The strategic part was objectives, metric definitions and ownership. The operational part was instrumenting the tracking with developers and building dashboards. We ran it as an agile programme, with a Trello board for the programme and Jira for the delivery work with developers. New requests arrived every week, so the backlog kept moving, and I prioritised by dependency and by how many teams a piece of work would unblock. A metric was defined before it was implemented.

Dashboards were released one implementation at a time. After each one, people could see an operational dashboard, use it and tell us what was missing, and most of the refinement of the metrics happened at that stage. When a new implementation moved into business as usual, a short survey checked whether it met the need it had been built for.

The programme delivered its 8 projects with 90% stakeholder satisfaction, and adoption of analytics grew 70% year on year. Adoption is the figure I value most, because it shows that people started coming to the data to settle questions.

What I learned

Implementing a metric that nobody has agreed on moves the argument downstream. In a meeting the disagreement is visible and cheap to resolve. Inside a dashboard it looks like a data quality problem, and tracing it back to a definition takes much longer.

The second lesson came from releasing early. A definition agreed in a workshop is still a draft until people use it with real data, and the first dashboard was the cheapest place to find out what the draft had missed.

What I would do today

Creating a metric keeps getting easier. With Tag Manager templates and point-and-click tagging, a new event can be live in minutes. When an event takes minutes to create, the definition is the part of the work that needs an owner.

If I ran the same programme now, I would add four habits.

1. A written definition for every primary metric, with the business question it answers, the formula, the data source, the filters, the owner and the date it was agreed.
2. A change log that the dashboard shows. When a definition changes, the date appears on the chart, so nobody reads a step in the line as a change in the business.
3. No new event in the tracking plan unless it feeds a metric that has a definition.
4. A review of the secondary metrics every quarter, to retire the ones nobody opens.

Before your next dashboard

Take the headline metric of the dashboard you are about to build and ask two managers who use it to write down, separately, how it is calculated. If the answers differ, that conversation comes first, and the dashboard can wait a week.

A hole in the checkout funnel: fix the tracking before you redesign the page

At JD Sports Group the business was ready to redesign the checkout because the funnel looked broken. The first problem I found was a page nobody had tracked, and fixing it changed what the rest of the project could do.

The situation

I was Senior Digital & Marketing Data Analyst at JD Sports Group from 2021 to 2024. One project started with a clear request: the checkout funnel is broken and needs redesigning. Several teams had a view on what was wrong and what to change, and the pressure was to start changing things.

What I did

I started with the analysis. Compared with benchmark, the funnel numbers were off in a way that did not fit a checkout that had not changed recently. Following the drop led me to a page that had been added to the checkout at some point and never tracked. Customers were passing through a step the funnel could not see, which left a hole in the report.

We fixed the tracking first. That gave the teams a funnel they could trust, and it gave me the credibility to ask for more advanced tracking, which was approved.

The numbers then showed where people left the checkout, but they could not say why. I added a survey with Hotjar, the tool the team already had, and used the OpenAI API to clean and categorise the open-text answers. Payment problems came out as the main theme, backed by what customers had written.

Session recordings came next, but the volume was too high to turn into a priority list. So I changed approach and worked with Customer Care to put live chat on the checkout during working hours, on one condition: agents logged the reason for each conversation in Salesforce. That gave us the reasons directly from customers, every week.

There were competing requests. Marketing wanted Customer Care to run video calls with customers, and Customer Care wanted the chat on product pages. I argued for the checkout first by walking both teams through the funnel step by step, showing what data we already had at each stage and what each option would tell us that we did not know yet. The reporting stayed simple, with charts and a weekly report, and the teams started reusing it in their own meetings. Once the report was useful to them, the priority was no longer up for debate.

The result, and how it was measured

Checkout conversion improved by 4% (relative) after the changes. That figure comes from a before-and-after comparison against the baseline before launch. There was no A/B test, so the comparison cannot separate the effect of the changes from seasonality or from anything else that moved in the same period. When I present the number, that limit goes next to it.

The weekly report outlived the project, and Customer Care was still using it in their own meetings long after it closed. It also showed me the weak point. As Customer Care hired new people, the logging discipline dropped and the reason data became less reliable, so the reports kept coming out while their value went down. Retraining became a recurring task, built into how the report was maintained.

What I learned

A funnel is a data product, and the cheapest data quality control is checking that every step in it is measured before anyone draws conclusions from it. The epilogue taught me the second lesson: data that depends on people typing a reason into a CRM needs the same upkeep as a pipeline, because adoption work does not end at handover.

What I would do today

I would add four things.

• Tracking in the definition of done. A new checkout step does not go live until its event is in the tracking plan and a test confirms that the event arrives.
• An automated completeness check on the funnel. Pages inside the checkout path that receive sessions and send no funnel event should raise an alert.
• The LLM categorisation treated as a measurement instrument. Personal data comes out of the free text before it goes to an external API, the prompt and model version are recorded, and a sample of answers coded by a person checks that the categories still hold.
• The comparison agreed before launch: the baseline window, the change being measured and the list of other releases in the same period. If the traffic allows it, a holdout group as well.

Try this on your own funnel

Pick your highest-value funnel and list every page a customer can see between the basket and the confirmation page. Then check that each one sends an event. If the two lists do not match, that is the first finding to take to the redesign meeting.

Who should see the AI's advice first? Why we put nutritionists in the loop

In 2024 we started building Nutr(IA) Coach, an AI product for nutrition. The first big product decision had little to do with models or prompts. It was about who receives the output. As a team we decided that the AI would work for nutritionists, and that a nutritionist would review its suggestions before they reached anyone.

The decision

The obvious version of the product gave nutrition advice straight to the public. It has the most potential users and the shortest path to growth. We chose the other version: the nutritionist uses the AI to prepare and speed up their work, and the person who receives the advice deals with a professional, with the AI behind them.

We made that call as a team, and the reason was ethical. Advice about what someone eats touches their health, and a person reading a confident answer on a screen has no easy way to tell a good suggestion from a harmful one. A nutritionist can, and can also see what the screen does not: the person's history, their circumstances and the questions that never made it into the prompt.

What the frameworks say

The principle has a long history in AI ethics, and reading the frameworks helped us put words to what we already felt.

The World Health Organization's guidance on ethics and governance of AI for health (2021) sets out six principles. The first is protecting human autonomy: humans should remain in control of health-care systems and medical decisions. Its 2024 guidance on large multi-modal models applies the same principles to generative AI.

The EU's Ethics Guidelines for Trustworthy AI (2019) list human agency and oversight first among seven requirements, and describe human-in-the-loop as one way to achieve it.

The AI Act turns human oversight into a legal duty for high-risk systems (Article 14), and since 2 February 2025 its first obligations apply, AI literacy among them. I am not saying the Act classes our product as high-risk. The point is that regulators and health bodies reached the same conclusion we did: when the stakes are personal, a person stays in charge.

The hard part: a reviewer can still rubber-stamp

A human in the loop is only a safeguard if the human actually reviews. Automation bias is well documented: people tend to accept a system's suggestion, more so when it is usually right and they are busy. A 2012 systematic review in JAMIA found it in clinical decision support. A nutritionist who approves every draft with one click gives the same protection as no nutritionist at all.

So the design question moves from "is there a human?" to "does the human have the time, the information and a reason to disagree?" These are the checks I would apply to any human-in-the-loop design:

1. The professional can see why the AI made a suggestion and which data it used.
2. Editing a draft is as quick as accepting it, so changing it costs nothing.
3. The AI says what it cannot handle and passes those cases straight to the professional.
4. Responsibility is written down: the professional signs the advice, and the record shows it.

What an analyst brings to it

Human in the loop also creates a measurement point that a direct-to-consumer product lacks. Every suggestion is accepted, edited or rejected by an expert. Those rates say more about the quality of the AI than a satisfaction score, and a rise in one-click approvals is an early warning of automation bias. Before a feature goes live, I want to know how we will measure it, and this design gives us something worth measuring.