Correlation in a personal log, and its limits

When two lines in your log rise and fall together, consider it a question to check, not an answer. Your log is observational data. It can show that two things moved in the same period. It cannot show that one caused the other, and the record gets worse if you write it as though it did.

This article covers what a correlation in a personal log can and cannot tell you, three things that quietly distort it, and which fields keep a pattern checkable months later. It is about reading your own record carefully. It does not tell you what to do with any pattern you find.

What a correlation in your log can and cannot tell you#

Self-tracked data is what researchers call an n-of-1 observational study: one person, observed over time, with nobody assigning anything at random. A methods paper on this topic describes using such data to find possible cause-effect relationships that can later be tested. The word "possible" matters. An association in your log is a candidate, not a finding.

The same paper is direct about the limit. Causal claims from observational data depend on assumptions that cannot be tested from the data alone, and the author closes with a line attributed to the statistician Donald Rubin: "For objective causal inference, design trumps analysis." In plain terms, a cleverer chart does not rescue a log that was never set up to answer the question.

That's not a reason to stop looking at your numbers. It's a reason to label what you see. "Energy scores were higher in the weeks when sleep entries were longer" is a statement about your record. "Longer sleep raised my energy" is a claim the record does not support. If you want to see how to keep those variables honest, tracking sleep and energy as variables, not verdicts covers the logging side.

Confounders: what else changed that week#

A confounder is something that moves both of the things you are comparing. If a stressful month shortens your sleep and also lowers your mood scores, those two lines will track each other whether or not one affects the other. Travel, illness, a change in routine, a different work schedule and the season can all do this.

The paper above gives a plain example from its own author's data. Reliable measurements of diet were missing, which the author says likely confounded the relationships in both analyses. The lesson for a personal log is that a factor you did not record cannot be ruled out later. You can only rule out what you wrote down.

| What you see | Other things that may have moved | What the log would need | | --- | --- | --- | | Two metrics rise together | Travel, illness, a busy stretch | A dated note for each | | A metric changes after a routine change | The season, a new schedule, a missed week | Consistent entry times and gaps marked as gaps | | Two metrics move in opposite directions | A third measure you did not log | A short list of fields you commit to keeping |

The right-hand column asks for more record, not more analysis. That's the pattern throughout: the fix for a weak correlation is better context at the time of entry.

Consecutive entries are not independent#

Entries from the same person on neighboring days tend to resemble each other. A poor night is often followed by another one. A week of low scores sits next to another week of low scores. Statisticians call this serial correlation, and it means 30 daily entries are worth less than 30 independent measurements.

An editorial on single-person trial designs notes that power declines as serial correlation increases, and describes how planned designs use washout periods and repeated alternations to reduce its effect. It also draws a line between those designs and casual records: N-of-1 trials, it says, are "scientifically rigorous methods" that should be planned in advance, in contrast to case descriptions.

Your everyday log is closer to a case description than to a planned trial. That's fine, as long as you read it that way. A long run of similar days can make a pattern look stronger than it is. When you scan a chart, remember that the points are neighbors, not strangers.

Many metrics, many pairs#

The more fields you track, the more pairs of fields you can compare. Some of those pairs will line up in your record by chance alone. This is a general statistical point rather than a finding from the sources above, but it explains why scanning a dozen metrics for anything that matches tends to turn up something.

A systematic review of self-tracking reports the practical side of this. It notes that the tracking workload is heavy and that it is hard to know what to track and how, and it flags the risk of obsessive tracking. It also says few studies examine how people perceive and interpret their tracking data. In other words, the gap between collecting numbers and reading them well is real, and it is not well studied.

Two habits help. First, decide in advance which comparison you care about, and write it at the top of the log with the date. A pattern you were looking for deserves a different label than one you found while scanning. Second, keep the number of fields small enough that you will fill them in. If that sounds familiar, the case for tracking less, more consistently makes the same argument from the data-quality side.

Record the context, not just the metric#

A pattern stays checkable when the log carries enough context to question it later. For each entry, these fields do most of the work:

• Date and time, in a consistent format, so neighbors in the chart really are neighbors.
• Units, written out (mg, mL, hours, a 1 to 5 scale) so a change in notation is not mistaken for a change in the data.
• Source, meaning where the number came from: a device, a lab report, a self-rating.
• A short note for anything unusual: illness, travel, poor sleep, a missed day, a change in routine.
• Gaps marked as gaps. A blank is information; a guessed number is noise.

If you use OptimizedTracker, the notes field is the place for the last two. The same applies to any spreadsheet. For more on building entries that stay readable, see turning daily check-ins into signal.

When a pattern belongs with your clinician#

If a pattern in your log seems to matter for your health, bring the record to a clinician rather than a conclusion. They can see your overall picture, which a chart of your own entries cannot. A clean, dated log with units and notes is useful to them. A causal story built from it is not something the log can support. For lab values in particular, reading your own lab trends without fooling yourself goes through what to record and what to leave to the person reading the results.

The short version#

A correlation in a personal log is an observation about your record. It may point to something worth asking about, but confounders you did not write down, neighboring entries that resemble each other and the sheer number of pairs you can compare all make it weaker than it looks. Label patterns as patterns, record context at the time of entry, and keep the claims as small as the data. You can browse more articles like this on the blog.