Skip to content

Commit

Permalink
Added margins option to help file and unit tests
Browse files Browse the repository at this point in the history
  • Loading branch information
@droodman
droodman committed May 8, 2021
1 parent cdff01d commit 1b2fa4e
Show file tree
Hide file tree
Showing 4 changed files with 109 additions and 5 deletions.
2 changes: 1 addition & 1 deletion boottest.ado
View file
Original file line number Diff line number Diff line change
Expand Up @@ -496,7 +496,7 @@ program define _boottest, rclass sortpreserve
local coleq: coleq `b'
if "`:word 1 of `coleq''"=="_" local coleq
local colnames: colnames `b'
forvalues i=1/`df' {
forvalues i=1/`=`df'' {
local terms 0
local constraintLHS`i'
forvalues j=1/`=colsof(`R')' {
Expand Down
20 changes: 16 additions & 4 deletions boottest.sthlp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ individual constraint expression must conform to the syntax for {help constraint
{synopt:{opt stat:istic(t | c)}}specify statistic type to bootstrap; default is {it:t}{p_end}
{synopt:{opt r:eps(#)}}specifies number of replications for bootstrap-based tests; deafult is 999; set to 0 for Rao or Wald test{p_end}
{synopt:{opt nonul:l}}suppress imposition of null before bootstrapping{p_end}
{synopt:{opt marg:ins}}bootstrap current results from {cmd:margins}{p_end}
{synopt:{opt madj:ust(bonferroni | sidak)}}specify adjustment for multiple hypothesis tests{p_end}
{synopt:{opt l:evel(#)}}override default confidence level used for confidence set{p_end}
{synopt:{cmdab:svm:at}[{cmd:(}{it:t} {cmd:|} {it:numer}{cmd:)]}}request the bootstrapped quasi-t/z distribution, or numerators thereof, be saved in return value {cmd:r(dist)}{p_end}
Expand Down Expand Up @@ -102,12 +103,16 @@ hypothesis, a confidence set is derived:

{title:Updates}

{p 2 4 0}* Since the publication of Roodman et al. (2019), {cmd:boottest} has gained one significant feature: the option to perform the bootstrap-c,
{p 2 4 0}* Since the publication of Roodman et al. (2019), {cmd:boottest} has gained two significant features. The first is the option to perform the bootstrap-c,
which bootstraps the distribution of the {it:coefficient(s)} of interest (or linear combinations thereof) rather t/z/F/chi2 statistics. Standard theory favors the latter,
but Young (2019) presents evidence that the bootstrap-c (or "non-studentized" test) is more reliable, at least in instrumental variables estimation. And theory and simulation in
Wang (2021) favors the non-studentized test when instruments are weak (but strong in at least one cluster). The option
{cmdab:stat:istic(c)} invokes the feature.

{p 2 4 0}* The second major new feature is the {cmdab:marg:ins} option, which allows you to bootstrap results from the {cmd:margins} command. To use this feature,
run {cmd:boottest} immediately after {cmd:margins} and do not include any hypotheses before the comma in the {cmd:boottest} command line. {cmd:boottest} will treat
each marginal effect separately.

{p 2 4 0}* Version 2.0.6 of {cmd:boottest}, released in May 2018, introduced two changes that can slightly affect results. The default for {opt r:eps(#)}
is now 999 instead of 1000. And in computing percentiles in the bootstrap distribution, ties are no longer (half-)counted.
For exact replication of earlier results, an older version, 2.0.5, is available as a
Expand Down Expand Up @@ -220,9 +225,6 @@ By default, the null is imposed before bootstrapping (Fisher and Hall 1990; Davi
estimation, the null is imposed only on the second-stage equation. Thus, after {cmd:ivregress 2sls Y X1 (X2 = Z)}, imposing the null of X1 = 0 results in the equivalent of
{cmd:ivregress 2sls Y (X2 = X1 Z)}, not {cmd:ivregress 2sls Y (X2 = Z)}.

{pstd}{cmd:boottest}'s bootstrap-based tests are generally fast, though the WRE runs substantially slower than the others. The code is optimized for clustered standard errors and
runs most efficiently in Stata version 13 or later.

{pstd}
The wild and score bootstraps multiply residuals or scores by weights drawn randomly for each replication. {cmd:boottest} offers five weight distributions:

Expand Down Expand Up @@ -306,6 +308,10 @@ and {cmd:scoretest} facilitate this usage.

{phang}{opt nonul:l} suppresses the imposition of the null before bootstrapping. This is rarely a good idea.

{phang}{opt marg:ins} instructs {cmd:boottest} to beach result from a call to {cmd:margins}, which must just have been generated, as a linear combination of parameters,
and test the independent hypotheses that each is 0. {cmd:boottest} extracts the coefficients of these linear combinations from the r(Jacobian) return value of
{cmd:margins}. When using this option, do not include any hypotheses before the comma (or in the deprecated {cmd:h0()} option).

{phang}{opt madj:ust(bonferroni | sidak)} requests the Bonferroni or Sidak adjustment for multiple hypothesis tests. The Bonferroni correction is
min(1, n*p) where p is the unadjusted probability and n is the number of hypotheses. The Sidak correction is 1 - (1 - p) ^ n.

Expand Down Expand Up @@ -501,6 +507,12 @@ giving back through a {browse "http://j.mp/1iptvDY":donation} to support the wor
{phang}. {stata cnsreg wage tenure ttl_exp collgrad, constr(1) cluster(industry)}{p_end}
{phang}. {stata boottest tenure} // wild bootstrap test of tenure=0, conditional on ttl_exp=2, Rademacher weights, null imposed, 999 replications{p_end}

{phang}. {stata regress wage tenure ttl_exp collgrad south#union, cluster(industry)} {p_end}
{phang}. {stata margins south}{p_end}
{phang}. {stata boottest, margins} // bootstrap CI of average predicted wage for south = 0 and 1{p_end}
{phang}. {stata margins, dydx(south)}{p_end}
{phang}. {stata boottest, margins graphopt(xtitle(Average effect of south))} // bootstrap CI of average impact in sample of changing south from 0 to 1{p_end}

{phang}. {stata ivregress 2sls wage ttl_exp collgrad (tenure = union), cluster(industry)}{p_end}
{phang}. {stata boottest tenure, ptype(equaltail) seed(987654321)} // Wald test, wild restricted efficient bootstrap, Rademacher weights, null imposed, 999 reps{p_end}
{phang}. {stata boottest tenure, ptype(equaltail) seed(987654321) stat(c)} // same but bootstrap-c{p_end}
Expand Down
6 changes: 6 additions & 0 deletions unit tests.do
View file
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,12 @@ constraint 1 ttl_exp = .2
qui cnsreg wage tenure ttl_exp collgrad, constr(1) cluster(industry)
boottest tenure, nogr // wild bootstrap test of tenure=0, conditional on ttl_exp=2, Rademacher weights, null imposed, 999 replications

regress wage tenure ttl_exp collgrad south#union, cluster(industry)
margins south
boottest, margins nogr // bootstrap CI of average predicted wage for south = 0 and 1
margins, dydx(south)
boottest, margins graphopt(xtitle(Average effect of south)) nogr // bootstrap CI of average impact in sample of changing south from 0 to 1

qui ivregress 2sls wage ttl_exp collgrad (tenure = union), cluster(industry)
boottest tenure, ptype(equaltail) seed(987654321) nogr // Wald test, wild restricted efficient bootstrap, Rademacher weights, null imposed, 999 reps
boottest tenure, ptype(equaltail) seed(987654321) stat(c) nogr // same but bootstrap-c
Expand Down
86 changes: 86 additions & 0 deletions unit tests.log
View file
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,92 @@ Wild bootstrap-t, null imposed, 999 replications, Wald test, bootstrap clusterin

95% confidence set for null hypothesis expression: [.008769, .189]

.
. regress wage tenure ttl_exp collgrad south#union, cluster(industry)

Linear regression Number of obs = 1,855
F(6, 11) = 466.17
Prob > F = 0.0000
R-squared = 0.2744
Root MSE = 3.5578

(Std. err. adjusted for 12 clusters in industry)
------------------------------------------------------------------------------
| Robust
wage | Coefficient std. err. t P>|t| [95% conf. interval]
-------------+----------------------------------------------------------------
tenure | .0371771 .0144888 2.57 0.026 .0052874 .0690668
ttl_exp | .2672039 .0213283 12.53 0.000 .2202605 .3141472
collgrad | 3.152462 .3625251 8.70 0.000 2.35455 3.950374
|
south#union |
0#union | .4785269 .3349975 1.43 0.181 -.2587976 1.215851
1#nonunion | -1.438916 .2353751 -6.11 0.000 -1.956973 -.9208588
1#union | -.3029596 .6704026 -0.45 0.660 -1.778506 1.172586
|
_cons | 3.581387 .220271 16.26 0.000 3.096574 4.0662
------------------------------------------------------------------------------

. margins south

Predictive margins Number of obs = 1,855
Model VCE: Robust

Expression: Linear prediction, predict()

------------------------------------------------------------------------------
| Delta-method
| Margin std. err. t P>|t| [95% conf. interval]
-------------+----------------------------------------------------------------
south |
0 | 8.164246 .3020966 27.03 0.000 7.499336 8.829156
1 | 6.887295 .4175234 16.50 0.000 5.968332 7.806258
------------------------------------------------------------------------------

. boottest, margins nogr // bootstrap CI of average predicted wage for south = 0 and 1

Wild bootstrap-t, null imposed, 999 replications, Wald test, bootstrap clustering by industry, Rademacher weights:
0bn.south 1.south

t(11) = 27.0253
Prob>|t| = 0.0040

95% confidence set for null hypothesis expression: [7.553, 9.373]

Wild bootstrap-t, null imposed, 999 replications, Wald test, bootstrap clustering by industry, Rademacher weights:


t(11) = 16.4956
Prob>|t| = 0.0010

95% confidence set for null hypothesis expression: [6.206, 8.86]

. margins, dydx(south)

Average marginal effects Number of obs = 1,855
Model VCE: Robust

Expression: Linear prediction, predict()
dy/dx wrt: 1.south

------------------------------------------------------------------------------
| Delta-method
| dy/dx std. err. t P>|t| [95% conf. interval]
-------------+----------------------------------------------------------------
1.south | -1.276951 .2201867 -5.80 0.000 -1.761578 -.7923233
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

. boottest, margins graphopt(xtitle(Average effect of south)) nogr // bootstrap CI of average impact in sample of changing south from 0 to 1

Wild bootstrap-t, null imposed, 999 replications, Wald test, bootstrap clustering by industry, Rademacher weights:
1.south

t(11) = -5.7994
Prob>|t| = 0.0210

95% confidence set for null hypothesis expression: [-1.885, -.1558]

.
. qui ivregress 2sls wage ttl_exp collgrad (tenure = union), cluster(industry)

Expand Down

0 comments on commit 1b2fa4e

Please sign in to comment.