Wikipedia:Reference desk/Archives/Mathematics/2009 December 11

Welcome to the Wikipedia Mathematics Reference Desk Archives
The page you are currently viewing is an archive page. While you can leave answers for any questions shown below, please ask new questions on one of the current reference desk pages.

December 11

weighted mean and errors

I have a sample of data points x_i, each with an associated error Δx_i, now i would like to average the x_i's , weighteing the data by the its Δx_i (ie the data with the smallest error contributes the most to the average. Currently i have my weighting factors as w_i=1-(abs(Δx_i/ (sum of Δx_i))

Now, is this
a) Something that would actualy give me an meaningful result? (w_i will sum to 1 so i think it can just do sum of w_i * x_i)
b) Is there a better way of doing this kind of problem? (surely someones has done a theory or two on this problem)
c) If i wanted to calculate the error in the average, could i do something with error propagation/adding in quadrature (weighted somehow) or would i be confined to only being able to use the standard deviation as a measure of the error?
Thanks for any help--137.205.21.58 (talk) 14:47, 11 December 2009 (UTC)

Just for readability, $w_{i}=1-\left|{\frac {\Delta x_{i}}{\sum \Delta x_{i}}}\right|$, right? --PST 14:57, 11 December 2009 (UTC)

yes, thanks, should get round to learning latex commands at some point--137.205.124.72 (talk) 15:15, 11 December 2009 (UTC)

If you know the variances of the errors, then the standard thing is to make the weigths proportional to the reciprocals of the variances. That minimizes the mean squared error of estimation. But in just what sense you "know the errors" is not clear from your posting. Michael Hardy (talk) 21:28, 11 December 2009 (UTC)

original op here, the errors for each data point are a combination of the systematic/statistical errors associated with measuring the value, i'm using some one elses data set and they've quoted x with an error Δx--86.27.192.94 (talk) 23:03, 11 December 2009 (UTC)

Our meta-analysis article mentions a few relevant techniques, including the "inverse variance method" Michael Hardy discussed above. -- Avenue (talk) 08:31, 12 December 2009 (UTC)

Are the data points all supposed to approximate the same value? The reciprocal of variances is fairly reasonable in that case. Or is it more like something like a problem I was looking where the points themselves are all different and I had to give some average value to their aggregate? With that I used the variance of the points plus their intrinsic variance to weigh them which probably didn't distringuish the better points as much as theoretically one should but worked well in practice. Dmcq (talk) 21:51, 13 December 2009 (UTC)

Original op here, yes the data should be approximately the same, just for clarity for working this out using the reciprocal of the variances:

Calculate ${\bar {\Delta x_{i}}}$(average of errors)

Calculate ${\sum (\Delta {x_{i}}^{2}-{\bar {\Delta x_{i}}}^{2})}$ (get the sum of the variances)

Calculate $w_{i}={\frac {(\Delta {x_{i}}^{2}-{\bar {\Delta x_{i}}}^{2})}{\sum (\Delta {x_{i}}^{2}-{\bar {\Delta x_{i}}}^{2})}}$ (normalise)

op here i think what i just wrote wasnt worth the electrons its displayed on, and thinking about this, doesnt using the reciprocal variance mean were giving more weight to the points closest to the mean, rather than the smallest points? which if i'm weighting by the errors (small error good, big error bad) what i want? —Preceding unsigned comment added by 86.27.192.94 (talk) 19:33, 14 December 2009 (UTC)

What was being said is to use the following formulae:

${\bar {x}}={\frac {\sum {x_{i}/\sigma _{i}^{2}}}{\sum {1/\sigma _{i}^{2}}}}$

$s^{2}={\frac {1}{\sum {1/\sigma _{i}^{2}}}}$

Personally I think this tends to accentuate the ones with a small variance too much - perhaps there is some way of taking into account that the variance has its own variation but I've not thought about that too much. Dmcq (talk) 00:59, 15 December 2009 (UTC)

Non-linear ODE question

Hi guys, I've been trying to find an analytical solution to $F'''(\psi )-2F(\psi )F'(\psi )=0$ for about half a day now, to no avail. What's screwing with me is the non-linear term in there; even if I use the fact that $2F(\psi )F'(\psi )={\frac {d}{d\psi }}(F(\psi ))^{2}$ to try to simplify it to a linear ODE, I get a nasty elliptical integral that I can't evaluate. Any ideas how to tackle this one? Titoxd^{(?!? - cool stuff)} 17:25, 11 December 2009 (UTC)

There's a derivative on the last F, right? $F'''(\psi )-2F(\psi )F'(\psi )=0\,$ ~~ Dr Dec (Talk) ~~ 17:29, 11 December 2009 (UTC)

Yep. Titoxd^{(?!? - cool stuff)} 17:31, 11 December 2009 (UTC)

As you said: 2F·dF/dψ = d(F²)/dψ, so your equation becomes d³F/dψ³ − d(F²)/dψ = 0. It follows that d²F/dψ² − F² = k for some constant k. You should be able to solve this in terms of the Weierstrass ℘-function. ~~ Dr Dec (Talk) ~~ 17:41, 11 December 2009 (UTC)

That will make evaluating the boundary conditions painful... thanks, though. Titoxd^{(?!? - cool stuff)} 19:20, 11 December 2009 (UTC)

What would you need to do? The solution is quite nice. Given arbitrary constants k (from my last post), c₁ and c₂ we have:

${\frac {1}{6}}F(\psi )=\wp \left(\psi -c_{1};-{\frac {k}{3}},c_{2}\right).$ ~~ Dr Dec (Talk) ~~ 21:53, 11 December 2009 (UTC)

Also note the particular solutions F(ψ)=12(ψ-c₁)^-2 (I guess they may be limit cases of your general formula). --pma (talk) 11:01, 12 December 2009 (UTC)

More or less. The particular solution is F(ψ) = 6(ψ − c₁)⁻², and this corresponds to k = c₂ = 0. ~~ Dr Dec (Talk) ~~ 00:49, 13 December 2009 (UTC)

oh it's 6, thanks --pma (talk) 08:50, 13 December 2009 (UTC)