Compare similar tasks
Each participant will produce two pieces of content from different briefs, but of similar difficulty. You will use AI on one task and work without AI on the other. The order will be drawn before starting.
Rovemark Research · RVMR-001
Proposed study · registrations openDoes AI save companies time?
Let's compare content production with and without AI. The study will measure how much time teams spend to produce, review and correct each delivery, in addition to quality and final cost.
Protocol v2.0.0 · 15 out 2026
How the study will be carried out
Registration is open. Before collection, we will define with the companies the number of participants, duration and data analysis. The study has not yet published results.
Record the time for each step
Production, review and correction time will be recorded separately. Rejected deliveries also enter the account, as they consumed the team's time.
Evaluate delivery
The reviewer will check whether the content is correct, meets the brief, is clear and presents sources that can be checked. Delivery must meet all four criteria. The reviewer will receive the material without knowing whether AI was used.
Review data for publication
We will analyze results even when AI does not bring gains or increase work. Before publication, data will be reviewed and anonymized. Disclosing a company's name requires specific authorization.
What will be measured
- Time per approved delivery
- Sum of production, review and correction minutes, including rejected attempts, divided by the number of approved deliveries. Accounting will be done separately for AI and non-AI tasks. If no deliveries are approved, this indicator will have no value.
- Cost per approved delivery
- Cost of hours worked plus spending on API or tool, divided by the number of approvals. The calculation will only be presented when the necessary costs have been entered.
- Quality and failures
- Number of approved and rejected deliveries. Assessment criteria, reviewer, briefs and permitted tools will be defined prior to assignments.
- Paired comparison
- Median time variation between the two tasks of the same person, considering only complete pairs with both deliveries approved. This cut-off may favor those who had a good result in both conditions. Therefore, the analysis will also inform how many pairs were left out.
Sample, privacy and limits of interpretation
Participation is voluntary. The results will describe the participating companies and may not represent the market. Sample size, timing, closure rule and analysis per company will be agreed upon prior to collection. The sample size necessary to detect an effect with statistical precision has not yet been calculated.
The person can learn from one task to another, and the briefs can have different difficulties. The order draw reduces some of this problem, but does not eliminate it. The tool presents descriptive summaries. It does not calculate statistical significance or confidence intervals.
In the collection tool, only record codes and measurements. Client briefs, names and documents should stay out of it. Registration contacts are saved separately. Participation in the study and publication of data will have their own agreements.
Rovemark sells AI tools and also conducts this research. This commercial link will be informed in the publications. The protocol is published on this website without OSF registration, NIST certification, or peer review. Any changes to the method will be identified and justified.
Methodological references
Planning prior to analysis:Center for Open Science. Measurement validity and context of use:NIST AI RMF. Difference between perception and observed time:METR (2025) andlimits update (2026). The references above guide the method and come from external institutions.
Participating companies
Register your company
Enter the process your company wants to evaluate. We will contact you to understand the team's routine and agree on participation conditions. Registration will be evaluated before confirmation.
Collection instrument · USD v2
Task registration
The coordinator can use this form to record measurements for each task. Materials and assessments must be kept separately, with access restricted to the responsible team. Times are entered manually.
The records in USD are in this browser.
Data from this form is not sent to Rovemark. Use codes instead of names and export a backup when you're done. To register a business, use the registration form above.
3. Check the records
No observations recorded in this browser.
Disclosure
Publicity materials
Download the graphics of the studies mentioned, the texts for social networks and an email template. The kit includes the sources and an invitation to participate in the survey.
https://rovemark.co/en/research/ai-productivity-pilot
Occupational exposure to AI · ILO 2025
Observed time and perception METR 2025
Income and labor participation · NBER, review 2026Version 2.0.0 · Review the email template and configure sender, recipients and unsubscribe before sending. The buttons on this page are used to download or copy the material.