Talk:Ramp function

Smoothed versions?

I came to this page in hopes of finding a canonical "smoothed ramp" function. That is, something like convolving ramp(x)=max(x,0) by a standard Gaussian kernel. For my application I would up cobbling one together using max() and erf(), but that's kludgy. I feel like there's a cannonical one that shows up in a Bode plot that probably comes from the log of a ratio or something, but I wasn't thinking of it. Thoughts? —Ben FrantzDale (talk) 12:22, 24 October 2011 (UTC)

Aah: I was thinking of this:

$f(x)=\log \left(e^{x}(e^{-x}+1)\right)$

That works. —Ben FrantzDale (talk) 13:20, 24 October 2011 (UTC)

Simpler form: $\log(1+e^{x})$, like you see in a Bode plot. —Ben FrantzDale (talk) 03:21, 10 April 2014 (UTC)

That's known as the **softplus** function in neural net lingo --mcld (talk) 19:28, 30 December 2015 (UTC)

Proposed merge with Rectifier (neural networks)

It seems that the (standard) rectifier is really the same as the ramp function. I don't see a reason to have to separate articles for the same function. QVVERTYVS (hm?) 13:53, 28 September 2015 (UTC)

I don't know what should be my opinion on this. The two articles take very different perspectives. The neural-net article has much better referencing (this article has nothing but a link to mathworld). I've not heard colleagues referring to "ramp function" but maybe it is a common term in some circles. I think my vote would probably to merge the "Ramp function" content into the rectifier page, unless people can prove to me (via citations) that "ramp function" is more common currency --mcld (talk) 19:32, 30 December 2015 (UTC)

I would not recommend the merge. The rectifier function has become so central to neural networks, especially deep networks, that I think more people would search for it in the context of neural networks. — Preceding unsigned comment added by 165.225.36.116 (talk • contribs) 15:56, 11 January 2016 (UTC)

I would also recommend the merge. The ramp and (linear) rectifier are exactly the same, it makes no sense to have separate articles for the same mathematical function. Ramp function has been around for about one century, alongside with unit step and impulse (Dirac delta) functions. Rectifier name was adopted recently (5 years or so) by neural network community (for deep learning). Also "ramp function" is probably a better name since rectifier is a well known electrical device. — Preceding unsigned comment added by Danilobassi (talk • contribs) 18:07, 20 December 2016 (UTC)

System of equations or piecewise function?

Right at the beginning of the Definitions section, I found this:

• A system of equations:

  $R(x):={\begin{cases}x,&x\geq 0;\\0,&x<0\end{cases}}$

Isn't this just a piecewise function, instead of a system of equations? — Preceding unsigned comment added by 85.2.203.202 (talk) 12:18, 5 February 2017 (UTC)