The output generated by the ptc.py script has been validated for all historical years following 2014 by comparing it to results extracted from the online Kaiser calculator as described below. The following explanation assumes that you have read all the documentation listed above "Project Validation" on the Documentation page.
As described on the Historical State Data page, the second-lowest-cost Silver plan (SLCSP) premiums for 2014 used by the ptc.py script are unlikely to be completely accurate, so you should avoid specifying any tax filing units for 2014 in your own sample input files.
In years after 2014 for which historical data are available, the ptc.py script has been validated in the following manner:
- The make_ptc_sample.py script in the test subdirectory is used to generate one or more tax filing units for the specified year in each of the 499 ACA rating areas in the fifty states and DC. The attributes of each filing unit are randomly drawn from probability distributions specified in the script and are written to a sample input file, which we refer to below as the validation sample file.
- The kaiser_ptc.py script reads the validation sample file and writes results (in the ptc.py output format) obtained from specifying the attributes of each tax filing unit (and its embedded ACA coverage family) in the online Kaiser calculator and then scraping the premium tax credit and intermediate results produced by the Kaiser calculator.
- The ptc.py script reads the same validation sample file and writes its own results in the standard output format.
- The ptc_out_diff.py script is then used to compare the results in the output file produced by the kaiser_ptc.py script and by the the ptc.py script.
This validation approach is not completely satisfactory in that there is a circularity with respect to SLCSP premiums. The SLCSP premiums used in the ptc.py script are extracted from the online Kaiser calculator, so that part of the PTC calculation is not being independently verified with this validation procedure, even though other aspects of the PTC calculation are being independently validated.
For 2015, two validation sample files have been constructed: one with just one tax filing unit per rating area (sample_15_1 in the test subdirectory) and another with ten randomly specified tax filing units per rating area (sample_15_10 in the test subdirectory). So, the smaller validation sample file contains 499 tax filing units, while the second validation sample file contains 4990 different tax filing units.
There are only minor differences between the Kaiser and OSPC output files generated from these two 2015 validation sample files. Details on what differences were found and how they were dealt with can be found in the closed ticket on sample_15_1 differences [552e91169d] and in the closed ticket on sample_15_10 differences [2fd9a81a36].
In summary, there were a few 2015 differences in simulated Medicaid eligibility with the Kaiser calculator attributing Medicaid eligibility to a few, but not all, families with incomes in the range from 139 to 140 percent of the federal poverty level. And there were many small differences in the IRS Form 8962 "applicable figure" caused by the Kaiser calculator approximating the "applicable figure" function in a way that slightly overstates the "applicable figure" consistently across all levels of modified AGI as a percent of the federal poverty level (magi_pct). For example, in the Kaiser calculator the "applicable figure" for those with magi_pct between 300 and 400 is consistently 9.56 instead of the 9.50 percent specified in IRS Form 8962. In addition, the "applicable figure" function used in the Kaiser calculator apparently does not use integer values of magi_pct, and therefore, a few large differences occur for tax filing units with magi_pct values close to where the function experiences a discontinuity (for example, at 100 and 400 magi_pct). But overall, there is extensive and close agreement between the Kaiser calculator results and the results produced by the ptc.py script for 2015.
This is the last User Document. You can elect to stop reading the documentation here, or continue if one of the two conditions is true. If you want to learn about using a version-control system to increase your productivity in using the OSPC-ACA source code, or if you want to learn about how to contribute your enhancements to the OSPC-ACA source code, begin reading the Contributor Documents.