<div class="footer"> <nav class="navFooter"> <div class="leftrow"> <a href="https://gatesfoundation.org" target="blank"> <img src="./bmgf-logo-white.png" title="Bill & Melinda Gates foundation" alt="IDMlogo" class="bmgflogo" /> </a> <cite class="cite" title="No reproduction or distribution without written permission of Bill & Melinda Gates foundation. Attribution-Noncommercial-ShareAlike 4.0 License."> <div class="copyText">© 1999-2022 Bill & Melinda Gates Foundation.</div> <div class="copyText">All Rights Reserved.</div> </cite> </div> <div class="rightrow"> <div class="column"> <a href="https://www.gatesfoundation.org/Terms-of-Use" target="_blank" class="link" rel="noopener noreferrer">Terms of use</a> <a href="https://www.gatesfoundation.org/Privacy-and-Cookies-Notice" target="_blank" rel="noopener noreferrer" class="link">Privacy & Cookies Notice</a> <a href="https://www.gatesfoundation.org/about/contact/reporting-scams" target="_blank" rel="noopener noreferrer" class="link">Be Aware of Fraudulent Activity</a> </div> <a href="http://www.idmod.org" target="blank"> <img src="./idmlogo55.png" title="Institute for Disease Modeling" alt="IDMlogo" class="idmlogo" /> </a> </div> </nav> </div> <div class="footer"> <nav class="navFooter"> <div class="leftrow"> <a href="https://gatesfoundation.org" target="blank"> <img src="./bmgf-logo-white.png" title="Bill & Melinda Gates foundation" alt="IDMlogo" class="bmgflogo" /> </a> <cite class="cite" title="No reproduction or distribution without written permission of Bill & Melinda Gates foundation. Attribution-Noncommercial-ShareAlike 4.0 License."> <div class="copyText">© 1999-2022 Bill & Melinda Gates Foundation.</div> <div class="copyText">All Rights Reserved.</div> </cite> </div> <div class="rightrow"> <div class="column"> <a href="https://www.gatesfoundation.org/Terms-of-Use" target="_blank" class="link" rel="noopener noreferrer">Terms of use</a> <a href="https://www.gatesfoundation.org/Privacy-and-Cookies-Notice" target="_blank" rel="noopener noreferrer" class="link">Privacy & Cookies Notice</a> <a href="https://www.gatesfoundation.org/about/contact/reporting-scams" target="_blank" rel="noopener noreferrer" class="link">Be Aware of Fraudulent Activity</a> </div> <a href="http://www.idmod.org" target="blank"> <img src="./idmlogo55.png" title="Institute for Disease Modeling" alt="IDMlogo" class="idmlogo" /> </a> </div> </nav> </div> <div class="footer"> <nav class="navFooter"> <div class="leftrow"> <a href="https://gatesfoundation.org" target="blank"> <img src="./bmgf-logo-white.png" title="Bill & Melinda Gates foundation" alt="IDMlogo" class="bmgflogo" /> </a> <cite class="cite" title="No reproduction or distribution without written permission of Bill & Melinda Gates foundation. Attribution-Noncommercial-ShareAlike 4.0 License."> <div class="copyText">© 1999-2022 Bill & Melinda Gates Foundation.</div> <div class="copyText">All Rights Reserved.</div> </cite> </div> <div class="rightrow"> <div class="column"> <a href="https://www.gatesfoundation.org/Terms-of-Use" target="_blank" class="link" rel="noopener noreferrer">Terms of use</a> <a href="https://www.gatesfoundation.org/Privacy-and-Cookies-Notice" target="_blank" rel="noopener noreferrer" class="link">Privacy & Cookies Notice</a> <a href="https://www.gatesfoundation.org/about/contact/reporting-scams" target="_blank" rel="noopener noreferrer" class="link">Be Aware of Fraudulent Activity</a> </div> <a href="http://www.idmod.org" target="blank"> <img src="./idmlogo55.png" title="Institute for Disease Modeling" alt="IDMlogo" class="idmlogo" /> </a> </div> </nav> </div> <div class="footer"> <nav class="navFooter"> <div class="leftrow"> <a href="https://gatesfoundation.org" target="blank"> <img src="./bmgf-logo-white.png" title="Bill & Melinda Gates foundation" alt="IDMlogo" class="bmgflogo" /> </a> <cite class="cite" title="No reproduction or distribution without written permission of Bill & Melinda Gates foundation. Attribution-Noncommercial-ShareAlike 4.0 License."> <div class="copyText">© 1999-2022 Bill & Melinda Gates Foundation.</div> <div class="copyText">All Rights Reserved.</div> </cite> </div> <div class="rightrow"> <div class="column"> <a href="https://www.gatesfoundation.org/Terms-of-Use" target="_blank" class="link" rel="noopener noreferrer">Terms of use</a> <a href="https://www.gatesfoundation.org/Privacy-and-Cookies-Notice" target="_blank" rel="noopener noreferrer" class="link">Privacy & Cookies Notice</a> <a href="https://www.gatesfoundation.org/about/contact/reporting-scams" target="_blank" rel="noopener noreferrer" class="link">Be Aware of Fraudulent Activity</a> </div> <a href="http://www.idmod.org" target="blank"> <img src="./idmlogo55.png" title="Institute for Disease Modeling" alt="IDMlogo" class="idmlogo" /> </a> </div> </nav> </div> <div class="footer"> <nav class="navFooter"> <div class="leftrow"> <a href="https://gatesfoundation.org" target="blank"> <img src="./bmgf-logo-white.png" title="Bill & Melinda Gates foundation" alt="IDMlogo" class="bmgflogo" /> </a> <cite class="cite" title="No reproduction or distribution without written permission of Bill & Melinda Gates foundation. Attribution-Noncommercial-ShareAlike 4.0 License."> <div class="copyText">© 1999-2022 Bill & Melinda Gates Foundation.</div> <div class="copyText">All Rights Reserved.</div> </cite> </div> <div class="rightrow"> <div class="column"> <a href="https://www.gatesfoundation.org/Terms-of-Use" target="_blank" class="link" rel="noopener noreferrer">Terms of use</a> <a href="https://www.gatesfoundation.org/Privacy-and-Cookies-Notice" target="_blank" rel="noopener noreferrer" class="link">Privacy & Cookies Notice</a> <a href="https://www.gatesfoundation.org/about/contact/reporting-scams" target="_blank" rel="noopener noreferrer" class="link">Be Aware of Fraudulent Activity</a> </div> <a href="http://www.idmod.org" target="blank"> <img src="./idmlogo55.png" title="Institute for Disease Modeling" alt="IDMlogo" class="idmlogo" /> </a> </div> </nav> </div> <div class="footer"> <nav class="navFooter"> <div class="leftrow"> <a href="https://gatesfoundation.org" target="blank"> <img src="./bmgf-logo-white.png" title="Bill & Melinda Gates foundation" alt="IDMlogo" class="bmgflogo" /> </a> <cite class="cite" title="No reproduction or distribution without written permission of Bill & Melinda Gates foundation. Attribution-Noncommercial-ShareAlike 4.0 License."> <div class="copyText">© 1999-2022 Bill & Melinda Gates Foundation.</div> <div class="copyText">All Rights Reserved.</div> </cite> </div> <div class="rightrow"> <div class="column"> <a href="https://www.gatesfoundation.org/Terms-of-Use" target="_blank" class="link" rel="noopener noreferrer">Terms of use</a> <a href="https://www.gatesfoundation.org/Privacy-and-Cookies-Notice" target="_blank" rel="noopener noreferrer" class="link">Privacy & Cookies Notice</a> <a href="https://www.gatesfoundation.org/about/contact/reporting-scams" target="_blank" rel="noopener noreferrer" class="link">Be Aware of Fraudulent Activity</a> </div> <a href="http://www.idmod.org" target="blank"> <img src="./idmlogo55.png" title="Institute for Disease Modeling" alt="IDMlogo" class="idmlogo" /> </a> </div> </nav> </div>

This is an online tool to explore the effects of exposure heterogeneity on HIV vaccine efficacy, given a leaky vaccine.

A potential outcome of pathogen exposure heterogeneity (i.e. variation among individuals in the risk of getting infected) is that vaccine efficacy measured from a trial (i.e. the clinical efficacy) is lower than the biological vaccine efficacy (i.e. the per-exposure or per-contact vaccine efficacy). This distinction, between the per-exposure vaccine efficacy and the clinical efficacy of the same vaccine, is the focus of this tool. Many authors have explored this issue, e.g. Halloran et al., 1992; White et al., 2010; O'Hagan et al., 2013; Edlefsen, 2014; Coley et al., 2016; Gomes et al., 2016; Kahn et al., 2018.

Here we use a simple epidemic model with exposure heterogeneity to simulate an HIV vaccine trial. Our goals are to:

  1. Raise awareness of the distinction between per-exposure vaccine efficacy, clinical vaccine efficacy, and population vaccine effectiveness
  2. Assess if this effect might contribute to the difference between the RV144, HVTN 702, and HVTN 705 vaccine trial outcomes
  3. Assist in the design or interpretation of HIV prevention trials, from this exposure heterogeneity perspective, and
  4. Use this framework as a means to explore HIV infection risk; more specifically, to delineate differences among individuals in HIV risk that is due to variation in per-exposure infection probability, HIV prevalence in a contact network, or the number of sexual contacts.

The separate tabs in this R Shiny app include:

  1. Model description, showing the structure of the model and the parameters included.
  2. Initial example plots, showing how the model works and what simulated epidemic and trial outputs we focus on.
  3. Parameter sweeps, which allows you to compare the impact of multiple parameter values in the same plots.
  4. Model fitting example, which allows you to use the model to examine specific trial results.
  5. Model fitting output, which shows parameter combos that are consistent with a given trial result.

The plots below allow you to see how the model works. You can change individual parameter values and observe the resulting changes in infections, incidence, and clinical vaccine efficacy.

Figure 1. Cumulative infections in the placebo arms of a vaccine trial, for populations with homogeneous risk and heterogeneous risk. Note that the infections in the heterogeneous risk population accumulate faster early in the trial, as the high-risk individuals are infected.

Figure 2. Incidence in the placebo arm of a vaccine trial. As expected from the cumulative infections plot above, the incidence in the heterogeneous risk population decreases over the course of the trial.

Figure 3. Incidence in the placebo and vaccine arms of a trial, for populations with homogeneous risk and heterogeneous risk.

Figure 4. Clinical vaccine efficacy over time in two vaccine trial population trials, one with homogeneous risk and the other with heterogeneous risk. Remember that the per-exposure vaccine efficacy is the value of the epsilon slider on the left.

The plots below allow you to see how clinical vaccine efficacy changes over the course of a trial under a range of parameter values.

Figure 1. The effect of varying the size of the high-risk subgroup in a vaccine trial population. On the left is clinical vaccine efficacy measured using cumulative incidence. On the right is clinical vaccine efficacy measured using instantaneous incidence, or hazard.

Figure 2. The effect of varying the overall incidence in a vaccine trial population.

Figure 3. The effect of varying the per-exposure vaccine efficacy (epsilon) in a vaccine trial population.

Here we use our model to identify what combinations of lambda (the overall rate of infection), risk (the risk multiplier for the high risk subgroup), and epsilon (the per-exposure vaccine efficacy) can produce pre-specified vaccine trial outcomes.

By 'pre-specified outcomes' we actually mean 'target statistics' for a model calibration. We use only two target statistics:

1. incidence in the placebo arm of a vaccine trial; and

2. the clinical vaccine efficacy.

We use a parameter exploration method in which we try many different values for each model input parameter (the number of tries is controlled by the 'executions' slider, below) and record the model output for each combination. The best parameter combinations (i.e. the modes) are found by rejection sampling, based on minimizing the euclidean distance between the target parameters and the simulation results, followed by local optimization. What the contour plots show, then, are the combinations of parameter values that, when input into our vaccine trial model, result in model outputs (placebo incidence and clinical VE) that are closest to our target statistics.

What combinations of lambda (overall infection rate) and risk heterogeneity are consistent with specific clinical HIV trial VEs?

What combinations of per-exposure VE and risk heterogeneity are consistent with specific clinical HIV trial VEs?

What combinations of per-exposure VE and lambda are consistent with specific clinical HIV trial VEs?

HVTN 705 model fitting results

This table shows the results of our model fitting runs where we used HVTN placebo incidence and clinical efficacy as the target stats in our rejection sampling calibration runs.

Matrix of distances between model runs and target stats

Model structure

We model a vaccine trial using an SI deterministic compartmental model; this is a simple epidemic model that has two populations, the Susceptible (S) and the Infected (I). We start the model with all trial individuals in the S group; over time S individuals move into the I group as they become infected.

More details are on IDM's Github repo here.

Note that this is specfically a vaccine trial model and not a more commonly used epidemic model (e.g. there are no births or deaths, and no recovery); we are not modeling infections from the I to S compartments, but rather only infections from a theoretical, un-modeled, outside (non-trial) population. We do this with one parameter, lambda, which is the rate that individuals in S get infected (move to I). This model structure also removes the possibility of indirect effects from vaccination.

Parameters

beta
transmission rate (per contact)
c
exposure rate (serodiscordant sexual contacts per time)
prev
prevalence (prevalence of viremic individuals)
lambda
lambda = beta * c * prev (this parameter defines the rate of infection)
risk
risk multiplier
epsilon
per contact vaccine efficacy; vaccine-induced reduction in the risk of HIV infection from a single exposure

The infection rate per time step is a combination of population prevalence prev (of viremic individuals), the exposure rate (serodiscordant sexual exposure per time) c, and the transmission rate (per exposure) beta.

The per exposure effect of vaccination is epsilon; epsilon is not time-varying (the per-exposure vaccine effect does not decay over time) and assumes a homogeneous effect (does not vary by viral genotype or individual traits).

We include three subgroups in the heterogeneous exposure population: high, medium, and low exposure. In reality we never fully know the correct size of HIV risk subgroups (i.e. fraction of the population) or their relative contribution to overall incidence.

The risk parameter (the risk multiplier) is an amalgam of increases in transmission risk that could be due to higher per-contact transmission or infection risk, higher exposure rate (number of contacts), or higher prevalence of HIV viremia in partners. Individual risk of infection can vary for these separately or in combination. Note that exposure heterogeneity, i.e. variation among individuals in the risk of infection, is most commonly discussed as variation in the number of HIV exposures. It can also include variation in the individual per-contact risk of acquisition, perhaps due to co-infections, viral load in the source individual or host genetics. Our risk parameter only multiplies the baseline risk of the medium risk subgroup, and it possible (in the underlying code) for the low risk subgroup to have zero risk.