Wikipedia:Reference desk/Archives/Mathematics/2009 April 29

Welcome to the Wikipedia Mathematics Reference Desk Archives
The page you are currently viewing is an archive page. While you can leave answers for any questions shown below, please ask new questions on one of the current reference desk pages.

April 29

Evaluating very small natural logs

I am interested in the ratio of 2 probabilities, where the numerator is the probability associated with a value of x (x~), and the denominator is the Sum of the probabilities all possible values of x (Sigma x).

I apologise for not being able to show this in LaTex notation.

The problem is the the probability for all values of x is tiny and expressed as a natural log. e.g ln(P(x))=(-50000). I cannot see how to evaluate this function without calculating the exp. of each term, then taking the ratio. But the number is too small to be dealt with...can I use algebra to express the answer in terms of natural logs?

Thanks!

Ironick (talk) 10:02, 29 April 2009 (UTC)

There is something wrong here, you can't take the log of a negative number. Are you saying that you take the log of a small number (eg 10^-50) and the log of that is a large negative number. But again, a negative number isn't right for a probaility which should be between 0 and 1. Please clarify -- SGBailey (talk) 10:30, 29 April 2009 (UTC)

Sorry, my mistake. Edited for clarity (I Hope). The output of the natural log terms are around -50000. So the natural log of the true probability is -50000. Making the true probability too small to deal with. Thanks. Ironick (talk) 10:38, 29 April 2009 (UTC)

I understand you have some events, say x, y, z..., with probabilities Px, Py, Pz... respectively.

And the probabilities are not known explicitly, instead you have their logs, ie. values: Lx=log(Px), Ly=log(Py), Lz=log(Pz)... which are 'large negative numbers'.

And you say you are interested in ratios, say Px/Py or Px/Pz — is that right?

If so, utilize the most important property of logarithms, that they reduce multiplication to addition:

$\log(ab)=\log a+\log b\,\!$

and consequently division to subtraction:

$\log(a/b)=\log a-\log b\,\!$

Thus your ratios can be calculated by

Lxy = log( Px/Py ) = log( Px ) − log( Py ) = Lx − Ly

and finally

Px/Py = exp( Lxy ) = exp( Lx − Ly )

CiaPan (talk) 11:04, 29 April 2009 (UTC)

Isn't the sum of the probabilities of all possible values of x simply 1? That's part of the definition of probability. --Tango (talk) 11:07, 29 April 2009 (UTC)

I guess you're trying to calculate $\log(e^{x}+e^{y})$ where x and y are too small (or huge) to evaluate the exponentials directly. In that case, a good approximation is

$\log(e^{x}+e^{y})\approx {\begin{cases}y+e^{x-y}&x\ll y\\y+\log(1+e^{x-y})&x\approx y\\x+e^{y-x}&x\gg y\end{cases}}$

The second case is exact, and you can calculate it directly on your floating point unit when x ≈ y. The first case is obtained from the second by using the fact that $\log(1+z)\approx z$ when z is small. The third is obtained by swapping the variables in the second and doing the same thing. Chances are you can ignore the exponential part of the first and third formulas and just use

$\log(e^{x}+e^{y})\approx {\begin{cases}y+\log(1+e^{x-y})&x\approx y\\\mathrm {max} (x,y)&{\text{otherwise.}}\end{cases}}$

-- BenRG (talk) 11:50, 29 April 2009 (UTC)

@CiaPan. Thanks, but unfortunately Lx-Ly is not computable, as too small.

@Tango. Yes, but I am dealing with the probability of data given a set of parameters. Liklihood is a better word. So they won't add to 1.

@BenRG. Thanks, that helps a lot, I think that's the solution. 130.88.243.41 (talk) 12:41, 29 April 2009 (UTC)

I don't understand this: unfortunately Lx-Ly is not computable, as too small.
If both Lx and Ly are negative, then their difference is LESS THAN 'greater' of them; it may be either negative or positive, but its abs() value does not exceed that of Lx and Ly. So, if Lx and Ly are computable (and they are, as you have computed them), then their difference is computable, too.
Example: for given log values −500 and −503 (assume they are 'large') the difference is either 3 or −3, which is certainly smaller (by absolute value) than 500 and 503.

In case you mean something like Lx=−500, Lx=−500.0001 — you just need larger precision arithmetics for your computations.
CiaPan (talk) 14:17, 29 April 2009 (UTC)

Apologies. I misunderstood/misexplained. I thought you meant the diffence between exp(-50000) and exp(-60000) rather than 50k and 60k=10k. The issue is not that I need to compute the ratio of 2 likelihoods, but for example (a)/(a + b + c) where a, b and c are all very small likelihoods. Which can be broken down to 1/1+(b/a)+(c/a). I am interested in the relative values of (a)/(a + b + c) compared to (b)/(a + b + c) and (c)/(a + b + c). 130.88.243.41 (talk) 15:59, 29 April 2009 (UTC)

By "relative values" do you mean ratios? They all have the same denominator, so you can just ignore it. The ratio of the first two is just a/b. --Tango (talk) 18:18, 29 April 2009 (UTC)

Integral - confusion

I'm trying to find the antiderivative of sqrt(C²+x²)dx

So I tried x=iC cos(iy) giving dx/dy=Csin(iy) and y=-ln(C)+ln(-x+sqrt(C²+x²) )

Substituting y for x gives me

sqrt(C2-C2 cos2(iy)) C sin(iy) dy

ie

C2 sin2(iy) dy

ie

C2 1/2 ( 1-cos(2iy) ) dy

So the antiderivative is

C2 1/2 ( y - (1/2i)sin(2iy) )

Without going further is there a mistake (partly because when I use the alternative substitution x=iC sin(iy) I get a change of sign in the antiderivative

ie C2 1/2 ( -y - (1/2i)sin(2iy) )

(I've been looking at the whole thing for about a week and can't find where I'm going wrong - I'm sure I used the same method some time ago for the "length of curve of a parabola" which involved sqrt(1+4x²)and I don't remember having any big problems.HappyUR (talk) 18:11, 29 April 2009 (UTC)

If you don't want to just look at the first integral at List of integrals of irrational functions for a solution, the problem is mixing i and sqrt. Square root can be positive or negative and you have to choose the right one to solve the problem, here by checking by differentiating again. You might find it easier to use tan instead of sin or cos and then you don't need to drag i into the equation so to speak. Dmcq (talk) 18:45, 29 April 2009 (UTC)

Can you clarify a bit about the square root - did you mean in:

y=-ln(C)+ln(-x+sqrt(C2+x2) )

Here I must take positive root for a postive log)

or in converting

sqrt(C2-C2 cos2(iy)) C sin(iy) dy

to

C2 sin2(iy) dy 

Here I don't see what difference it makes whether I take a positive or negative root - it doesn't seem to solve the issue I had?HappyUR (talk) 19:07, 29 April 2009 (UTC)

I'm afraid I didn't look to closely. It could be either or both and also the log could have bits added too when you start getting into complex logarithms. However just looking at your first bit

So I tried x=iC cos(iy) giving dx/dy=Csin(iy) and y=-ln(C)+ln(-x+sqrt(C²+x²) )

I believe the expression for y is wrong. The transformation for arccos in Inverse trigonometric function#Logarithmic forms is:

$\arccos x=-i\,\ln \left(x+i\,{\sqrt {1-x^{2}}}\right)={\frac {\pi }{2}}\,+i\ln \left(i\,x+{\sqrt {1-x^{2}}}\right)={\frac {\pi }{2}}-\arcsin x=\operatorname {arcsec} {\frac {1}{x}}$

You'd have to use the version with the π/2 to get rid of the i in the ln. Dmcq (talk) 20:16, 29 April 2009 (UTC)

ok I need y = -i arcos (x/ic)

I think I missed a +ln(i) from my expression for y ( ln(i)=pi/2+2npi or something doesn't) it, if I add that to my expression for y I think its still right (It doesn't affect the integral when evaluated, I was aware of that) if you

- as I said I really have been doing this for a week and I'm starting to go a bit blind.. I'm sure it's an obvious typo or missed sign. But I'm at a can't see the wood for the trees state now - which is why I ask - typos aside - is the substitution ok ? (I'm 100% sure it worked before - my head is actually starting to hurt..) Will thank profusely anyone who can sort this out for me. ThanksHappyUR (talk) 21:24, 29 April 2009 (UTC)

I still haven't looked right through the calculation carefully, but just on an off chance, you're not getting confused because you've used y for two different things and the results look rather similar? Dmcq (talk) 10:45, 30 April 2009 (UTC)

Another thing, where you say you have to take the positive sqrt to get a positive log in ln(-x+sqrt(C²+x²)), thazt isn't true. If you've got i around the place one could have ln(x+sqrt(C²+x²)) + ln(-1). Complex logs are allowed to work on negative or complex numbers. Dmcq (talk) 10:53, 30 April 2009 (UTC)

There's only one type of y I've used, also in my equation - there is no i in the roots - if you look at the equation for y (it's at the top) (both C and x are real). Please read question before answering.HappyUR (talk) 14:50, 30 April 2009 (UTC)

Unless I'm misreading there is one y in x=iC cos(iy) and another different y in x=iC sin(iy). Dmcq (talk) 16:53, 30 April 2009 (UTC)

Plus as to the business about being real what you should have written I believe is x=C cos(iy) for the first substitution if you want both x and y to be real. The i is okay in the second. Also sinh doesn't cover the whole range of reals which could be another problem but not what I think is happening here. Dmcq (talk) 17:10, 30 April 2009 (UTC)

Let me clarify - I used the substitution x=iC cos(iy) and didn't get the right antiderivative, I also tried the subsitution x=iC sin(iy) and got a different (change of sign) antiderivative

So I have two problems - not getting the right antiderivative and getting different antiderivatives depending on substituion - I must have made more than one mistake?

I need to use x=iC cos(iy) for the substitution to work if I use x=C cos(iy) it gets me nowhere.

y is not real - it's complex. I don't need y to be real?HappyUR (talk) 17:57, 30 April 2009 (UTC)

Please see Hyperbolic function. cos(iy) is the same as cosh(y) and -i sin(iy) is the same as sinh(y). If using C cos(iy) went nowhere it was possibly because cosh as a real function is always greater than or equal to 1. Just sticking in an extra i won't make both x any y real, the sinh version should work okay though. Putting in i's and using sin though confuses it quite a bit. Dmcq (talk) 09:15, 1 May 2009 (UTC)

For god's sake I already said y is NOT REAL. I know that. I said that in the last sentence before your post: "y is not real - it's complex". I'm asking where the mistake is in the working or reasoning. The reason I don't use C cos(iy) is because I'm inserting an i to get the substituted equation to be sqrt(C²+i²cos²(iy) ) which is C sin(iy). I've noticed that I don't need to use iy, I could just use cos(y) because y is naturally a complex number in this example. Are any of the steps I first described at the top wrong? —Preceding unsigned comment added by HappyUR (talk • contribs) 12:32, 1 May 2009 (UTC)

Would you please consider again what I said about your using two different y's. You define y in two different ways. One of the y's is real when x is real. The other is not. I believe you are confusing the two when you say you have two different integrals. They look similar but different because the initiual substitutions are simililar but different. The rest is all about the messing about with i's and your argument about the ln because of reals is wrong but that only affects later on when substituting the value of the y's in to each. Dmcq (talk) 18:23, 1 May 2009 (UTC)

Yes ok, so, at the beginning of this post - in which I only deal with one subsitution ie upto the point "...Without going further is there a mistake (partly because..." in between which steps did my mistakes happen?HappyUR (talk) 03:14, 2 May 2009 (UTC)

No real problems. Theres a few things about introducing functions that don't have unique inverses and it's unnecessary bringing in complex numbers but that's minor. Ok sorry I see you just wanted assurance nothing much had gone wrong up to that point and forget about anything else. Dmcq (talk) 09:57, 2 May 2009 (UTC)

ok When I replace y and sin(2iy) in the antiderivative with the respective x functions, and then differentiate with respect to x I don'y seem to be getting the original function

Must be a mistake in my differentiation, I'll look again at that Thanks.HappyUR (talk) 12:45, 2 May 2009 (UTC)