How to Design Impactful Health Outcomes Research Studies for Analysis

Jacob VanHouten, clinical data analyst and biostatistics consultant at Kolabtree, provides top tips on how to design impactful health outcomes research for analysis. 

If you are in the medical field and have read, done, or even thought about doing research within the past two decades, it is likely you are familiar with the concept of outcomes research. In a nutshell, outcomes research is a field where the object of study is a definable health outcome, which I will give examples of later, and what is measured are the effects that different parts of the overall workflow have on those outcomes. As the point of such studies is to determine what factors affect the outcomes of interest, many different experimental and observational study designs are well-suited for this task, including classic epidemiologic studies such as:

• randomized controlled trials,
• cross-sectional studies,
• cohort studies,
• systematic reviews/meta-analyses, or
• quality improvement research methodologies.

Why even do outcomes research?

There are many, many reasons to do outcomes research, and listing them all would be far beyond what would be possible to do in this post. In a super-small nutshell, outcomes research as it applies to the field of biomedicine is primarily focused on improving the performance of some task within said field. As famed management consultant Peter Drucker is credited with saying, “You can’t manage what you can’t measure.” In other words, you cannot “move the needle” on improvement without knowing where it is pointing in the first place. Additionally, such measurement can lead to

• quality improvement,
• decreased healthcare costs,
• improved efficiency of diagnosis and treatment, and
• patient experience.

And who wouldn’t want the best outcomes for their patients? It is arguably healthcare’s primary research imperative to be continually measuring for improvement, and outcomes research is one powerful set of tools by which to get there.

How to think about outcomes research like a statistician

When thinking about starting a research endeavour, perhaps the first thing to know is “what do you want to know?” Are you more interested in the system and efficiency, or on the intangible quality of life measurements as determined by the patients? Are you interested in making care more affordable, equitable, and accessible to patients, or is your concern the profitability of a practice? Some key questions include:

• what is the outcome of interest,
• who are the relevant stakeholders,
• how is the outcome of interest best represented, and
• how can I get the data I need to answer my questions?

Types of outcomes

One of the strengths of outcomes research is the ability to consider many different outcomes and their relative merits, as well as from many different points of view (more on that below). In fact, some outcomes research constructs, such as Quality-Adjusted Life Years, have been designed specifically to make the comparison of different outcomes. Outcomes which may be of disparate types can be compared through conversion of one outcome into an equivalent (i.e., amount of money one would have to receive to give up one night of sleep) outcome that is more directly comparable.

Who cares?

When thinking about the above outcomes to measure, maybe the first question should be “who cares?”. And this is not meant flippantly. Sincerely, who is it that cares about this outcome. The patients? The providers? Insurers, health systems? It is not unreasonable to imagine that a patient and a hospital value the outcome of the patient’s satisfaction very differently, though it is important to both. To adequately account for the outcomes you want to measure, you must consider whose perspective(s) are the best from which to evaluate. Clearly identifying at the beginning of an analytic plan the perspective from which you will consider the outcomes protects against both confusion and post hoc data manipulation, whether accidental or not. While a  comprehensive list of potential outcomes of interest is beyond the scope of this article, the following table highlights some of the most widely used categories of outcomes.

Widely used categories of outcomes

Safety	Misuse of medical therapy and oversight in the course of clinical care; Medical mistakes that place patients at risk for adverse events
Effectiveness	The gap between what can be achieved through medical intervention or policy and what is actually accomplished
Equity	Examination of disparity in healthcare delivery that focuses on whether nonclinical factors such asrace,gender, andsocioeconomic status influence the care of patients
Efficiency	Focuses on ways to maximize efficiency, limit healthcare costs, and reduce waste in the healthcare system.
Timeliness	Patient access to healthcare: barriers to access, and uninsured patients inability to benefit from healthcare.
System responsiveness	Educational efforts amongst the medical community and implementation of healthcare policies that improve patient care
Patient-centeredness	How medical interventions will affect patients, what patients feel and what they can do to effect medical decision making.

 

Data types

In addition to the kind of outcomes you are interested in, it is worth thinking about the way in which you conceive of data, specifically in terms of data types.

Data come in two main flavors: numeric, and categorical.

Numerical data

Numeric is just as it sounds; the variable being measured is quantitative, being of either the type integer, which are the whole numbers, and floats, which are all the numbers with some non-whole number part.

Examples of integers include number of babies delivered in a hospital, results of a Likert scale questionnaire on patient satisfaction, or number of minutes taken during a surgery, as well as many, many others.

Categorical data

Categorical data are those data that can only take certain specific values. Some data points are categorical and dichotomous, meaning the variable can take one and only one of two possible outcomes.  For example, a light bulb can either be turned off, or on, but it will be one of those options, and not the other. Sometimes there are more than two categories, and this defines a nominal variable. Nominal variables have multiple different possible values, but no natural ordering among them; an example might be flower types, where the plant may be a rose, tulip, daisy, sunflower, etc. Finally, categorical variables that do have a natural ordering but are still restricted to specific outcomes are termed ordinal.

An example of this type of variable may be a categorical representation of patient satisfaction: unsatisfied, slightly satisfied, satisfied, very satisfied. Even those there is a restricted universe of possible outcomes, these levels have a natural ordering between themselves.

The reason it is so important to be aware of outcome types and data types is because you will largely be deciding how to model the data yourself, which will in turn determine which types of analyses are possible. If you want to know the number of surgical operations in your hospital per day, you may use the actual integer number (1,2,3,etc), or you could factorize them into high, medium, and low volume days. In the end, the way you choose to represent the data reveals to those reading your work how you view the world as well as why you made the decisions you made. They may not agree with you, or be able to reproduce your data, but if you leave no room for ambiguity, there is no question regarding the truth of your findings.

Where do your data come from? Are you going to collect them or obtain them from another source? If you are recording the data yourself, the responsibility is yours to decide what you will record and what you will not, which will impact the available analysis options. If you are not going to collect it yourself, how is the data set currently stored (data type, location, etc.)? And very importantly: know and understand the process by which the data are produced and collected. Misunderstanding on these issues can lead to research that does not answer its intended question.

Bonus tip: Why hiring a statistician could save your study

I’m not a car guy. When mine needs regular work or a specific repair, I’m the first guy to take it to the shop. Why? Because I know that I do not have the skills to do the job. Similarly, not everyone will be doing their own statistics, either because they do not have the required training or simply because they choose to put their efforts elsewhere. With that in mind, those who still wish to do outcomes research but do not want to be responsible for their own analysis should consider hiring a freelance statistician, the likes of which you are able to find easily on Kolabtree.

If you do choose to work with a statistician, do yourself a favor and get them involved earlier rather than later. As the famed (if not kindly remembered) statistician R.A. Fisher is quoted, “To consult the statistician after an experiment is finished is often merely to ask him to conduct a post-mortem examination. He can perhaps say what the experiment died of.”

This is absolutely true, in that once an experiment has been run, and the data collected, there are some analysis methods that are no longer available that may have been if different decisions had been made at earlier stages of research.

In addition to not having to do your own statistical analysis, there may be other tangible and intangible benefits to working with a statistician. For instance, they likely have through their training been exposed to some more complex methods of experimental design or analysis, and it is possible that using one of these rather than standard methods could significantly save resources such as time, participants, or money. There may also be new ideas in the field which you may not be aware of, such as best practices for reproducibility of findings or the most up to date software packages for complex analyses. Best of all, right now could be the best time to snag an excellent statistician for a bargain. Given the chilling economic effects of the pandemic, individuals across all walks have been hit hard. Statisticians affected by the pandemic are looking for freelance gigs, and many are willing to give either discounts in exchange for loyalty.

Conclusion

This is by NO MEANS an exhaustive discussion of outcomes research; rather, it should serve as a bite-sized intro for complete novices. But, even for such researchers, a little up-front thought regarding your outcome of interest, how the data elements will be represented, and where you will be able to get the data can go a long way towards making sure that the outcomes research you perform is meaningful and answers the question for which you intend it. And remember, if you feel you can’t or would prefer not to do the analysis yourself, or if you’d like to learn more about the newest analysis methods available, don’t forget to seek out/reach out to your statistics colleagues.

Need help with conducting a clinical trial and analyzing outcomes? Consult a clinical research consultant on Kolabtree or work with a freelance statistical analyst.