{smcl}
{hline}
 help for {hi:kapci}{right:(SJ4-4: st0076)}
{hline}

{title:Confidence intervals for the kappa statistic}

{p 8 16 2}
{cmd:kapci} 
{it:varlist} 
[{cmd:if} {it:exp}] 
[{cmd:in} {it:range}] 
[{cmd:,} {cmdab:est:im(an bc p n bsall)}
{cmdab:w:gt(w w2} {it:user_wgt}{cmd:)}
{cmdab:r:eps(}{it:#}{cmd:)}
{cmdab:si:ze(}{it:#}{cmd:)}
{cmdab:se:ed(}{it:#}{cmd:)}
{cmdab:ev:ery(}{it:#}{cmd:)}
{cmdab:l:evel(}{it:#}{cmd:)}
{cmdab:t:ab}
{cmd:wide}
{cmdab:sa:ving(}{it:filename}{cmd:)}
{cmd:replace}
{cmdab:nom:sg} ]

{p 4 4 2}
{cmd:by} ... {cmd::} may be used with {cmd:kapci}; see help {help by}.


{title:Description}

{p 4 4 2}
{cmd:kapci} calculates the confidence interval (CI) for the kappa statistic of 
interrater agreement using an analytical method in the case of dichotomous 
variables (Fleiss 1981) or bootstrap for more complex situations (Efron
and Tibshirani 1993; Lee and Fung 1993). 

{p 4 4 2}
Computer efficiency is the main advantage of using an analytical procedure.
Alas, to best of the author's knowledge, no such method has yet been developed
to accommodate more complex analysis beyond the simple 2x2 case. Although
quite computationally intensive and time consuming, the bootstrap method
may be an interesting option to calculate confidence intervals when variables
have more than two levels; if three or more raters/replications are involved;
and, above all, when a weighted kappa is envisaged. As a compromise between
efficiency and necessity, {cmd:kapci} implements a specially developed
subroutine to handle the analytical solution or Stata's {help bs} 
({hi:[R] bstrap}) program for the bootstrap when required.

{p 4 4 2}
Details about the bootstrap procedure can be found in Stata's 
documentation. As for the sub-routine, it is based on the asymptotic variance 
developed by Fleiss, Cohen, and Everitt (1969; Fleiss  1981, equations
13.15-13.18). In {cmd:kapci}, calculations use an algebraically modified
formulation directly taken from Stata's {help sskdlg} (Reichenheim 2000),
which is based on Cantor (1996). 


{title:Options}

{p 4 8 2}
{cmd:estim(}{it:estimid}{cmd:)} requests the type of confidence interval to
be displayed.  The following {it:estimid} are available: {cmd:an}
(analytical), {cmd:bc} (bias corrected), {cmd:p} (percentile), and {cmd:n}
(normal). The first is only suitable (it is the default) when the data is
dichotomous or two raters/replications are involved. Otherwise, bootstrap is
needed, and any of other three may be chosen. {cmd:bc} is the default. All
three bootstrap estimations may be displayed at once with option {cmd:bsall}.
    
{p 4 8 2}
{cmd:wgt(}{it:wgtid}{cmd:)}  specifies the type of weight that is to be used
to weight disagreements. This option is ignored if there are more than two
raters/ replications (varlist >= 3). As in @kap@, user-defined weights can be
created using {cmd:kapwgt}. However, {cmd:wgt(w)} uses the "prerecorded"
weights 1- |i-j|/(k-1), where i and j index the rows and columns of the
ratings by  the two raters and k is the maximum number of possible ratings.
{cmd:wgt(w2)}  uses the "prerecorded" weights 1-((i-j)/(k-1))^2.

{p 4 8 2} 
{cmd:reps(}{it:#}{cmd:)} specifies the number of bootstrap replications (B)
to be performed.  This option is ignored if {cmd:estim(an)} is requested or if
the data is dichotomous and option {cmd:estim()} is omitted. The default
number of bootstrap replications has been set to 5 for syntax testing only. In
general, {cmd:reps()} must be increased when analyzing real data.

{p 4 8 2}
{cmd:size(}{it:#}{cmd:)} specifies the bootstrap size of the samples to be
drawn. The default is _N, meaning that samples are drawn the same size as the
data. This option is ignored under the same conditions as {cmd:reps()}.

{p 4 8 2}
{cmd:seed(}{it:#}{cmd:)}  specifies the initial value of the random-number
seed used by {cmd:bs} running under {cmd:kapci} when bootstrap is
requested or needed. This option is useful for reproducibility of results.
{it:#} should be specified as an integer. {cmd:seed()} is ignored under the
same conditions as {cmd:reps()}.

{p 4 8 2}
{cmd:every(}{it:#}{cmd:)} specifies that results be written to disk
every #th replication.  This option should be specified only in conjunction
with {cmd:saving()} when performing bootstraps that take a very long time. This
will allow recovery of partial results should the computer crash; see help
{cmd:postfile} (manual:  [P] postfile). {cmd:every()} is ignored under the
same conditions as {cmd:reps()}.

{p 4 8 2}
{cmd:level(}{it:#}{cmd:)} specifies the confidence level, as a percentage,
for the confidence interval.  Default is {cmd:level(95)}.

{p 4 8 2}
{cmd:tab} displays all possible two-way tabulations of the assessments.

{p 4 8 2}
{cmd:wide} requests the display of wide two-way tables. Unless the option is
specified, wide tables are broken into pieces to enhance readability. This
option is ignored if {cmd:tab} is omitted.

{p 4 8 2}
{cmd:saving(}{it:filename}{cmd:)}  dumps B bootstrapped kappas to
{it:filename}{cmd:.dta} on the {cmd:pwd}.  if {cmd:by} ... {cmd::} is 
specified, dumping goes to separate files according to by-groups in the
by-variable.  As many files ({cmd:.dta}) are created as there are by-groups,
which are indexed accordingly from 1 to k and respecting the ascending order
of values (e.g., {it:filename}{cmd:1.dta}, {it:filename}{cmd:2.dta}, ...,
{it:filename}{k.dta}).

{p 4 8 2}
{cmd:replace} allows overwriting a file that already exists on the {cmd:pwd}
when using {cmd:saving()}.

{p 4 8 2}
{cmd:nomsg} suppresses the printing of a warning message that is automatically
displayed when {cmd:reps} > 100.


{title:Saved Results}

{p 4 4 2}
{cmd:kapci} saves in {cmd:r()}:

{p 8 22 2}{cmd:r(kappa)}{space 3}point-estimate kappa statistic{p_end}
{p 8 22 2}{cmd:r(se)}{space 5}   standard error{p_end}
{p 8 22 2}{cmd:r(z)}{space 6}    {it:z}-score{p_end}
{p 8 22 2}{cmd:r(N)}{space 6}    sample size{p_end}


{p 4 4 2}   Available only if analytical estimation is requested

{p 8 22 2}{cmd:r(prop_e)}{space 2}expected proportion (agreement){p_end}
{p 8 22 2}{cmd:r(prop_o)}{space 2}observed proportion (agreement){p_end}
{p 8 22 2}{cmd:r(ub_an)}{space 3}analytical upper confidence interval{p_end}
{p 8 22 2}{cmd:r(lb_an)}{space 3}analytical lower confidence interval{p_end}


{p 4 4 2}Available only if bootstrap estimations are requested

{p 8 22 2}{cmd:r(ub_bc)}{space 3}bias-corrected upper confidence interval{p_end}
{p 8 22 2}{cmd:r(lb_bc)}{space 3}bias-corrected lower confidence interval{p_end}
{p 8 22 2}{cmd:r(ub_p)}{space 4}percentile upper confidence interval{p_end}
{p 8 22 2}{cmd:r(lb_p)}{space 4}percentile lower confidence interval{p_end}
{p 8 22 2}{cmd:r(ub_n)}{space 4}normal upper confidence interval{p_end}
{p 8 22 2}{cmd:r(lb_n)}{space 4}normal lower confidence interval{p_end}
{p 8 22 2}{cmd:r(bias)}{space 4}bias{p_end}
{p 8 22 2}{cmd:r(reps)}{space 4}number of bootstrap replications (B){p_end}
{p 8 22 2}{cmd:r(N_bs)}{space 4}bootstrap size of the samples to be drawn{p_end}

 
{title:Examples}

{p 8 12 2}{cmd:. kapci meas1_G2 meas2_G2, tab}

{p 8 12 2}{cmd:by byvar: kapci meas1_G2 meas2_g2}

{p 8 12 2}{cmd:. kapci meas1 meas2, w(w2) r(1000) se(123456) tab wide estim(bsall) sa(filename.dta) replace}


{title:Author}

{p 4 4 2} Michael E. Reichenheim, IMS/UERJ, Brazil, michael@@ims.uerj.br


{title:References}

{p 4 8 2}
  Cantor, A. B. 1996. Sample size calculations for Cohen's k. 
  {it:Psychological Methods} 1: 150-153.

{p 4 8 2}
  Efron, B. and R. Tibshirani. 1993. {it:An Introduction to the Bootstrap}. 
  London: Chapman & Hall.  
  
{p 4 8 2}
  Fleiss, J. L., J. Cohen, and B. S. Everitt. 1969. Large sample standard
  errors for kappa and weighted kappa. {it:Psychological Bulletin} 72: 323-327.

{p 4 8 2}
  Fleiss, J. L. 1981. {it:Statistical Methods for Rates and Proportions}.
  2nd ed. New York: Wiley.

{p 4 8 2}
  Lee, J. and K. P. Fung. 1993. Confidence interval of the kappa coefficient
  by bootstrap resampling [letter]. {it:Psychiatry Research} 49: 97-98.

{p 4 8 2}
  Reichenheim, M. E. 2000. sxd3: Sample size for the kappa-statistic of
  interrater agreement. {it:Stata Technical Bulletin} 58: 41-45.
  In {it:Stata Technical Bulletin Reprints}, vol. 10, 382-387. 
  College Station, TX: Stata Press.
  

{title:Also see}

{p 4 13 2}
STB:{space 4} STB-58 sxd3

{p 4 13 2}
Online:  help for {help kap}, {help kappa}, {help sskdlg} (if installed),
{help concord} (if installed), {help alpha}, {help bs}, {help bstrap}
                   
{p 4 13 2} Manual: {hi:[R] kappa}