The basic logic of Non-linear Modeling: Binary Outcomes

Non-linear regression is very similar in spirit to linear regression and the general linear model. In this situation we are dealing with binary linear outcome, and we are going to have to end up modeling it on a transformative, non-linear scale. So what that means is we will start with something that is measured as a yes or no, one or zero, and then transform it to a probability. And the thing we're going to estimate is a linear function of our predictors, transformed into a non-linear outcome. Might sound a bit confusing, but just hang in there.

Regression with continuous dependent variables

Lets say you're all of a sudden interested in the attendance to baseball games. You want to understand what affects the level of attendance to baseball games, lets call it ATTEND (in thousands of people), and you believe that a current wins in the season have a say in how the level of attendance to the baseball games look. In other words, you believe there is a linear relationship between attendance to a baseball team's games and the wins they have in a season. This is called a linear model, and in that case, we can notate the model like this:

or simply...

Based off this linear relationship, estimating this model is fairly easy. You can observe the relationship between the two using methods of summary statistics, like scatter plots, and based on the apparent upward trend in the plot points, you may infer that there is a positive relationship between the X and Y variables, probably leaving you with the question of how strong and significant the magnitude of this relationship is.

With this particular example, you're bound to receive regression estimates that confirm the positive relationship between the two variables.

 Dependent variable: ATTEND

              coefficient    std. error   t-ratio   p-value
  -----------------------------------------------------------
  const        −1340.92      387.127      −3.464    0.0009    ***
  CURNTWIN      38.6325      4.74786       8.137    6.02e-012 ***

And when you interpret the magnitude of the coefficients, you can conclude that for every unit increase in CURNTWIN, ATTEND sees an increase by 38.6325. In laymen's terms, for every additional win a baseball team has, their stadiums are expected to experience an increase in attendance by about approximately 38 to 39 people.
OLS models are very popular approaches to regression analysis, and in these models we're dealing with dependent variables that have continuous values.

Regression with Dichotomous Dependent variables

Lets change that up and assume that instead of baseball wins and attendances, you want to find out the probability that a person attends college given the percentage level of their parental wage. We can just write that as...

 or simply...

We're going to move away from the linear combination of the independent variables (LCIV) because they actually give off nonsensical results. For instance, a regression equation with a continuous variable like income can provide useful information when you generate graphs, like a scatter plot, but with dichotomous variable as Y, we get something where all the plot points are scattered to either one value or the another, like this:

The points are bounded to either 0 or 1, which makes a lot of sense (if you're familiar with dummy variables) because 0 and 1 represent outcomes (0 ="didn't attend college" ; 1 = "attended college").

So the rectification is to take a non-linear transformation of the linear combination of independent variables, looking like this:

The properties we expect function F to have reflect the discrete outcomes of interest (as related to qualitative variables). In that role, function F is expected to output 0 as the linear combination of independent variables tend towards negative infinity, and F is expected to output 1 as the LCIVs tend towards positive infinity. That's just a mathematical way of stating this hypothetical point, that as we see a continual increase (or decrease) in the value(s) we get from the LCIVs, we should also gradually expect the outcome of interest (Y) to be a likely occurrence (or likely nonoccurrence). 

The corresponding math notations usually looks like this:

F(-∞) = 0

F(+∞) = 1

and thus, this transformation of the independent variables means that our probability belongs in the range of 0 and 1, or...

In context, lets assume that we see the value for X as very high, indicating high parental income; in that case, then we can assume that the probability of a person going to college is virtually 1, or very likely to happen.

Conclusions

The basic logic of logit and probit modeling is grounded on this reasoning, and will often give similar results in their estimations. Instead of trying to see the effects of the X var(s) on the continuous Y variable, like the effects of income on overall consumption, you're trying to see the how the X var(s) influences the probability that dichotomous Y variable will occur; in other words, the likelihood of outcomes.

It should be understood and reminded that yes, we can expect a higher or lower probability of an event occurring but that does not mean it will indeed happen or not. The ultimate outcome can most definitely go the other way. 

Note: Regression equations always include error terms, which were purposely excluded in this post. Just know that you are to always include the error term when conducting analyses for academic or professional work.

Sources:

Heavily borrowed and influenced from:

• YOUTUBE: Discrete choice models - introduction to logit and probit