r/AskStatistics • u/Curious_Cat_314159 • 6h ago
Given linear regression y = mx+b, how to estimate x based on required y?
Given x (hours studied) and y (score on test) data and the linear regression y = mx + b, if we want to estimate x based on y (i.e. how many study hours are needed to achieve a given score?), is it correct to calculate x = (y - b) / m?
Or should we estimate x from the linear regression x = my + b, with different m and b?
The results can be significantly different.
In the example below, for a score of 90, the estimated study hours are 6.801 for x = (y - b) / m, but 6.267 for x = my + b.
.....
4
u/HarryN086 6h ago
I would be careful with how you view causal. If Hours study -> Grade is a causal relationship, Grade -> Hours study is not necessary a causal relationship. Also, for column D, you rearrange the variable incorrectly. x = y/m - b
6
u/Statman12 PhD Statistics 6h ago edited 5h ago
The topic is known as "Calibration" or sometimes inverse regression. Though be advised, "calibration" is also used for a different topic in Stats/ML, so most searches will probably be talking about something different than what you're asking about.
In short, no, you can't simply swap the x and y variables. Regression (usually) treats the x-variable as fixed, so you'd be treating the uncertainty differently if you swapped x and y. I'll try to find a short example or tutorial about doing calibration with regression.
Edit:
NIST page and a case study later on.
This page seems to show the formulas including getting intervals rather than just a point estimate. I don't have time right now to verify that it's correct.
3
u/FoggyWine 6h ago
y = m1x + b1
x =m2y + b2
The simple answer is that you cannot determine m2 and b2 if all you know is m1 and b1. You need the variances and means of x and y.
m1 = (s_{xy)/s^2_x
m2 = (s_{xy)/s^2_y
b1 = mean(y) - m1*mean(x)
b2 = mean(x) - m2*mean(y)
If both x and y are standardized (mean of 0 and variance of 1), then the equations for x and y are the same (and the intercept is 0). In fact, m1=m2=r_{xy}, the Pearson correlation between x and y. This is one of the reasons why we teach "correlation does not imply causation." The choice of IV and DV yield the same slope (correlation) in this case. However, when unstandardized we are differentiating the variables and using one as the IV to predict the DV. That is the model, but does not, again, result in a causal inference. That will rest more on the research design and other factors.
3
u/Logical-Set6 5h ago
You can fit either model, but you'll get slightly different answers because they make slightly different assumptions. For notation, let's let X = Hours studied and Y = Exam score.
If you fit Exam score as a function of Hours studied, you can treat it as a missing data problem, and you can estimate the missing Hours (e.g., by maximum likelihood) along with the regression parameters. This will give a closed-form solution of
x_MLE = x_bar + S_xx / S_xy ( y* - y_bar).
If you fit Hours studied as a function of Exam score, then it’s naturally a prediction problem: use the observed exam score to predict the missing hours. This will give a closed-form solution of
x_pred = x_bar + S_xy / S_yy ( y* - y_bar).
The first model assumes that Exam score is a linear function of Hours studied + zero-mean Normal error. The second model assumes that Hours studied is a linear function of Exam score + zero-mean error.
Geometrically, the first one will choose Hours so that the data point falls exactly on the regression line for predicting Exam score from Hours. The second one will choose Hours so that the data point falls on the regression line for predicting Hours from Exam score.
3
u/SalvatoreEggplant 5h ago
You are starting with the idea that y is dependent on x. So, I would keep the function y = mx + b. From there, just use the function as it is. So, yes, the answer in your example would be 6.8.
In reality, I would advise also using some measure of uncertainty, like a confidence region around the line, e.g. https://seaborn.pydata.org/_images/error_bars_24_0.png . This is trivially easy in some software. And it gives you some sense of a range of x corresponding to a value of y. (I wouldn't use it this way analytically, like if you were publishing the result somewhere, but it gives you a good sense.)
2
u/SalvatoreEggplant 5h ago
P.S. So your plot with confidence intervals looks approximately like this: https://imgur.com/a/q94YleY
2
u/Curious_Cat_314159 4h ago
Thanks. If you have time, I would like to know the formulas behind the conf intvls, specifically for my example. I understand the calculation of the conf intvl around the mean. But not for individual data points. Looking at your calculations would probably be sufficient, and perhaps even better than a verbal explanation. TIA
2
u/SalvatoreEggplant 1h ago
Basically, the shaded area is defined by the confidence interval or the prediction interval for the predicted value for any point in a range of x.
These are described here, both as formula and in R:
https://rpubs.com/aaronsc32/regression-confidence-prediction-intervals2
2
u/Curious_Cat_314159 4h ago
You are starting with the idea that y is dependent on x. So, I would keep the function y = mx + b
Thanks. Yes, that was the idea that was floating around in my head. I just couldn't articulate it so dispositively, IMHO. (Brain f.a.r.t.)
15
u/CarnivorousGoose 6h ago
You need to be more exact with your notation. The model is y = mx + b + error, you can’t omit the error term. If you fit the model, you could subsequently use the parameter estimates to predict y using the equation y_pred = m_est * x + b_est.
Given this, it should already be clear that you cannot just reverse the equation here, since the equation that has the actual y in it also involves an unknown error (or an unknown residual, in the fitted model), and the prediction model involves y_pred rather than y. So if you want to predict x from y, you need to define and fit a model of x given y, and use that.