{"text_format":"2","workbook":"xls","n_sheets":3,"sheets":["CAA metrics","CWA metrics","RCRA metrics"],"not_rendered":{"charts":0,"images":0,"macros":0,"external_links":1,"defined_names":6,"comments":0,"pivot_tables":0},"unopened":[],"bounds":{"document_bytes":16777216,"sheet_cells":100000,"run_cost":"one","rows_first":2000,"rows_last":500,"rows_between":2000,"halvings":0,"cell_chars":16384,"cell_head_chars":4096},"cells_cut":0,"cells_raw":0,"read":"whole","date_system":"1900","formulas":"cached_only"}

{"sheet":"CAA metrics","index":0,"n_rows":85,"n_cols":13,"rows_kept":85,"cut":false,"hidden":false,"merged":[[6,2,6,12],[2,2,2,12],[3,2,3,12],[4,2,4,12],[5,2,5,12],[16,2,16,12],[17,2,17,12],[18,2,18,12],[19,0,19,1],[61,0,62,0],[61,1,62,1],[1,0,1,1],[1,2,1,12],[2,0,6,1],[7,0,7,1],[8,0,12,1],[13,2,13,12],[14,2,14,12],[15,2,15,12],[79,0,80,0],[70,10,72,10],[65,1,66,1],[65,0,66,0],[70,0,72,0],[70,1,72,1],[67,0,69,0],[67,1,69,1],[0,0,0,12],[81,0,82,0],[81,1,82,1],[77,0,78,0],[73,1,75,1],[79,1,80,1],[77,1,78,1],[73,0,75,0],[13,0,13,1],[14,0,18,1],[21,0,21,11],[32,0,32,11],[31,2,31,10],[22,0,23,0],[25,0,26,0],[34,0,34,11],[36,0,36,11],[41,0,41,11],[44,0,44,11],[48,0,48,11],[67,12,69,12],[70,12,72,12],[73,12,75,12],[64,1,64,10],[50,0,50,11],[53,0,53,11],[55,0,55,11],[59,0,59,11],[63,0,63,11],[22,12,23,12],[25,12,26,12],[61,12,62,12],[77,12,78,12],[79,12,80,12],[81,12,82,12],[7,2,7,12],[8,2,8,12],[9,2,9,12],[10,2,10,12],[11,2,11,12],[12,2,12,12],[65,12,66,12],[19,2,19,12]],"flags":["formulas_cached_only"]}
["CAA Metrics - Please answer the following four questions."]
["1.",null,"What is your name and contact information?  "]
[null,null,"Name:  "]
[null,null,"Organization:  "]
[null,null,"Address:  "]
[null,null,"E-Mail Address:  "]
[null,null,"Phone Number:  "]
["2.",null,"What new metrics are needed? (Please also explain how the proposed metric would support your program evaluation.)"]
[null,null,"1.  "]
[null,null,"2.  "]
[null,null,"3.  "]
[null,null,"4.  "]
[null,null,"5.  "]
["3.",null,"Which metrics should we consider removing? (Please provide a brief justification.)"]
[null,null,"1.  "]
[null,null,"2.  "]
[null,null,"3.  "]
[null,null,"4.  "]
[null,null,"5.  "]
["4.",null,"Do you have any suggestions for improving the existing metrics?  (Please list suggested improvements in the far right column of the table below (\"Your Suggestions\").  Please type each of your suggestions in the yellow shaded region next to the corresponding metric.)"]
[null,"Name","Description","Guidance Requirement or Goal","Universe","Combined Region/State or State-only Data?","Original Source of Data Provided by EPA","Selection Criteria Explanation","Data Pulled From","Metric Type","Notes","Your Suggestions"]
["1. Degree to which state program has completed the universe of planned inspections/compliance evaluations (addressing core requirements and federal, state and regional priorities) is completed.",{"$nulls":11},1]
["a","1a1. CAA major full compliance evaluation (FCE) coverage in last two complete fiscal years","Percent of CAA active major sources receiving full compliance evaluations by the state in the last two fiscal years.  Data pull uses the current universe. ","Active major sources should receive a full compliance evaluation every 2 years. If region and state negotiated less than 100% 2-year coverage, refer to 1r below.","AFS Majors *See note in Selection Criteria Explanation.","State-only (but separate combined % also provided)","AFS","Select all major CAA sources from AFS which have had at least one FCE action within the last two complete fiscal years. Compare to the universe of all active majors plus other inspected majors. *(Numerator counts any AFS major source with an FCE regardless of operating status/denominator counts all AFS major sources with operating status O,T, or I, plus any other inspected majors.)","IDEA ","Goal","Consistent with CMS. The select logic for this metric could be improved in the future when AFS \"CMS category\" and \"automatic unknown compliance flag\" data are quality tested as a way to measure inspection commitments. Note re: universe counts: OECA researching option of capturing End-of-Year universe starting with FY2005 for use where universe counts are required. If sources included on the list were not active for the full two years, FCE is not expected. ",null,"a"]
[null,"1a2. CMS major full compliance evaluation (FCE) coverage in last two complete fiscal years","Percent of CMS major sources receiving full compliance evaluations by the state in the last two fiscal years.  Data pull uses the current universe. ","CMS sources should receive a full compliance evaluation every 2 years. If region and state negotiated less than 100% 2-year coverage, refer to 1r below.","CMS Majors *See note in Selection Criteria Explanation.","State-only (but separate combined % also provided)","AFS","Select all major CMS sources from AFS which have had at least one FCE action within the last two complete fiscal years. Compare to the universe of all active CMS majors plus other inspected CMS majors. *(Numerator counts any CMS major source with an FCE regardless of operating status/denominator counts all CMS major sources with operating status O,T, or I, plus any other inspected CMS majors.)","IDEA","Data Quality","During the time frame of the review, states/locals will continually refine the universe of affected CMS sources.  Therefore, both the AFS and CMS universes should be used for comparison purposes.  Utilizing both universes allows a QA/QC review to identify any major sources that have not been included in CMS. If sources included on the list were not active for the full two years, FCE is not expected."]
["b","CAA synthetic minor 80% sources (SM-80) FCE coverage","Percent of CAA currently active SM-80 sources with full compliance evaluations by the state in the last four <increasing to five in FY06> fiscal years. Metric will be a 5-year 100% goal when data are available.","Active SM-80 sources should receive a full compliance evaluation every 5 years.  However, the CMS policy has only been effective since FY02, so a 2002-2005 metric is provided (80% goal) (increasing to 2002-2006 at 100%). If region and state negotiated less coverage, also see 1r.","CAA SM-80%s *See note in Selection Criteria Explanation.","State-only (but separate combined % also provided)","AFS","Select all CAA SM-80 sources from AFS which have had at least one FCE within four <increasing to five in FY06> complete fiscal years.   Compare to the universe of all active 80% synthetic minors plus other inspected SM-80s. *(Numerator counts any CAA SM-80 source with an FCE regardless of operating status/denominator counts all CAA SM-80 sources with operating status O,T, or I, plus any other inspected SM-80s.) ","IDEA","Goal","Consistent with CMS.  Current NPG document does not discuss whether regions and states can combine to meet goal.   The select logic for this metric could be improved in the future when AFS \"CMS category\" and \"automatic unknown compliance flag\" data are quality tested as a way to measure inspection commitments. Note re: universe counts: OECA researching option of capturing End-of-Year universe starting with FY2005 for use where universe counts are required. ",null,"b"]
["c","1c1. FCEs and reported PCEs for currently active synthetic minors (coverage)","Informational measure provides data for all synthetic minors. ","FCEs at federally reportable sources are required to be reported. PCEs that are part of a negotiated compliance monitoring plan are to be reported pursuant to the Compliance Monitoring Strategy (CMS). ","AFS SMs *See note in Selection Criteria Explanation.","State-only (but separate combined % also provided)","AFS","Select all CAA SM sources from AFS which have had at least one FCE or reported PCE within the last four fiscal years <increasing to five in FY06>. Compare to the universe of all active AFS SM sources plus other inspected AFS SMs.  *(Numerator counts any SM source with an FCE or reported PCE regardless of operating status/denominator counts all SM sources with operating status O,T, or I, plus any other inspected SMs.) ","IDEA (Partially available in OTIS.)","Informational-only.","Although state PCEs that are not included in a negotiated CMS plan and FCEs at non-federally reportable sources are not required, they will be included for informational purposes when voluntarily reported. This metric helps provide a complete picture of state evaluation activity.  Note re: universe counts: OECA researching option of capturing End-of-Year universe starting with FY2005 for use where universe counts are required. Note: state PCEs are not required but will be included for informational purposes if reported.",null,"c"]
[null,"1c2. FCEs and reported PCEs for CMS synthetic minors (coverage)","Informational measure provides data for all synthetic minors. ","FCEs at federally reportable sources are required to be reported. PCEs that are part of a negotiated compliance monitoring plan are to be reported pursuant to the CMS. ","CMS SMs *See note in Selection Criteria Explanation.","State-only,  (but separate combined % would also be provided)","AFS","Select all CMS SM sources from AFS which have had at least one FCE or reported PCE within the last four fiscal years <increasing to five in FY06>. Compare to the universe of all active CMS SM sources plus other inspected CMS SMs.  *(Numerator counts any CMS SM source with an FCE or reported PCE regardless of operating status/denominator counts all CMS SM sources with operating status O,T, or I, plus any other inspected CMS SMs.) ","IDEA","Informational-only","Although state PCEs that are not included in a negotiated CMS plan and FCEs at non-federally reportable sources are not required, they will be included for informational purposes when voluntarily reported. This metric helps provide a complete picture of state evaluation activity. Also, during the time frame of the review, states/locals will continually refine the universe of affected CMS sources. Therefore, both the AFS and CMS universes should be used for comparison purposes. Utilizing both universes allows a QA/QC review to identify any SM sources that have not been included in CMS. Note: state PCEs are not required but will be included for informational purposes if reported."]
["d","FCEs and reported PCEs at minor sources (coverage)","Informational measure provides data for all minor sources. ","Although not all minor source activity or PCEs are required to be reported, this metric provides useful information on the overall health of the compliance monitoring program. ","AFS minor sources *See note in Selection Criteria Explanation.","State-only","AFS","Select all CAA minor sources from AFS which have had at least one FCE or reported PCE within the last four FYs <increasing to five in FY06>. Compare to the universe of all active AFS minors plus other inspected AFS minors. *(Numerator counts any AFS minor source with an FCE or reported PCE regardless of operating status/denominator counts all AFS minor sources with operating status O,T, or I, plus any other inspected AFS minors.) ","IDEA","Informational-only","Some state/locals have either negotiated or expressed interest in substituting minors for larger sources, or evaluate them in addition to the CMS sources.  Although not all minor source activity is reportable, this metric provides useful information on the overall health of the compliance monitoring program. OECA will output the CMS flag, which states may use to reflect negotiated minors into their CMS plans. Note: state PCEs are not required but will be included for informational purposes if reported.",null,"d"]
["e","Investigations at CAA stationary sources","Informational measure provides information on this type of activity consistent with CMS. ","Required pursuant to the current Air Facility System (AFS) Information Collection Request (ICR). Category of compliance monitoring activity recognized by CMS. ","Active AFS sources","State-only","AFS","Select all active sources from AFS which have had at least one investigation conducted (e.g., achieved date) within the last four fiscal years <increasing to five in FY06>.","IDEA","Informational-only","Investigations are a category of compliance monitoring activity recognized by CMS.  They are more resource-intensive than the other compliance monitoring activities, and provide useful information on the overall health of the compliance monitoring program.",null,"e"]
["f","Review of self-certifications completed","% of self certifications received by state in fiscal year that have been reviewed.","CMS recommends all self-certifications to be reviewed.  Goal = 100%.","All active majors with Title V Air Program Code 'active', and cert due, rec'd or rev'd.","State-only","AFS","Select all active CAA Title V sources from AFS which have had at least one \"Compliance Certification State Review\" action within the fiscal year.   Compare to the universe of active CAA Title V sources with any certification due/received or reviewed action within the FY.","OTIS CAA Extended Management Reports (one state at a time)","Goal","High percentages shown (that is 100%) may be significantly affected by incomplete reporting of certification due/received MDR actions by the regions or states (if negotiated).  Percentage is also affected by the fact that some reviews are for certs received in the previous FY. All majors supposed to have Title V permit by 2006.",null,"f"]
["g","Number of sources with unknown compliance status","AFS is designed to convert compliance status to unknown based upon the FCE frequency negotiated between the region and state.  Frequency is variable, thus this measure may more precisely track whether commitments are completed.","CMS provides for flexibility in FCE frequency, which is tracked using AFS.","All majors and SM-80s with CMS (refer to 12j)","Combined","AFS","Select all sources where the current source compliance status is \"unknown\" as automatically generated by AFS.","OTIS CAA query","Review Indicator","Will pull current data.",null,"g"]
["r","Percent of planned FCEs/negotiated PCEs completed","States/locals, with regional approval, can develop alternative CMS plans.  Such alternative plans can vary with respect to the universe of sources covered, the minimum frequencies, and/or the type of evaluation (i.e., FCE, PCE or investigation).  To the extent possible, AFS should be used to document and track progress with these alternative plans.  If not possible, regions and states/locals must still provide sufficient documentation to track and evaluate progress with these plans.",{"$nulls":9},"r"]
["2. Degree to which inspection (Compliance Monitoring) reports document inspection findings, including accurate description of what was observed to sufficiently identify violations.",{"$nulls":11},2]
["a","Percentage of Compliance Monitoring Reports adequately documented in the files.","Evaluation of documentation in the file that is based on the elements in the EPA Compliance Monitoring Strategy.",null,null,"State-only",null,null,null,"Review Indicator"]
["3. Degree to which inspection (Compliance Monitoring) reports are completed in a timely manner, including timely identification of violations.",{"$nulls":11},3]
["a","Percentage of Compliance Monitoring Reports which identify potential violations in the file within a given time frame established by the Region and state, within 60 days.","Evaluation of documentation in the file that is based on the elements in the EPA Compliance Monitoring Strategy.",null,null,"State-only",null,null,null,"Review Indicator"]
["4. Degree to which significant violations (e.g., significant noncompliance and high priority violations) and supporting information are accurately identified and reported to EPA national databases in a timely manner. ",{"$nulls":11},4]
["a","High priority violation discovery rate - (per FCE coverage at majors)","New HPVs identified in fiscal year by the state divided by the number of sources with state FCEs performed in the fiscal year.","File review recommended for regions/states below 1/2 of the national average. (HPV policy)","Majors","State-only (with Regional-only provided)","AFS/ IDEA","Select major sources with new HPVs identified in the FY.  Compare to the number of major sources with State FCEs conducted (onsite or offsite) within the FY.","IDEA ","Goal","Regional-only numbers will show the HPV discovery rate for regional inspections within each state. Accurate HPV identification is important for timeliness tracking under the HPV policy, the Watch List, award program screening, and public access to the data.",null,"a"]
["b","High priority violation discovery rate (per source universe - major)","New HPVs identified in fiscal year by the State divided by the number of major sources.","File review recommended for regions/states below 1/2 of the national average. (HPV policy)","Majors","State-only (with Regional-only provided)","AFS","Select major sources with new HPVs identified in the FY.  Compare to the number of active major sources.","IDEA ","Review Indicator","Assists region in selecting files for review when there is a possible issue.",null,"b"]
["c","No activity indicator - HPV","No HPVs identified by the state in the fiscal year.","HPV policy","All AFS","State-only","AFS","Select sources with new State HPVs identified in the FY.  ","OTIS Management Reports","Review Indicator","Assists region in selecting files for review when there is a possible issue. Region should examine state enforcement cases to determine whether HPVs were not identified.",null,"c"]
["d","HPV reporting indicator at majors (formal enforcement only)","Metric computes the % of formal actions taken at major sources during the FY that received a prior HPV listing, and benchmarks it to national average.","Targeted file review, in addition to SRF random selection, recommended for regions/states less than half of the national average.","Majors with formal actions","State-only","IDEA or AFS","Select the universe of major CAA sources that received a formal enforcement action within the FY.  Compare to the subset of these that were designated as an HPV (source flag, not pathway) within the same FY or the two quarters prior to that FY.","IDEA","Review Indicator","Assists region in selecting files for review when there is a possible issue; for example, if a low percentage of actions received an HPV listing, the region should review files at facilities without such HPV listing.",null,"d"]
["5. The degree to which state enforcement actions include required corrective or complying actions (injunctive relief) that will return sources to compliance in a specific time frame.",{"$nulls":11},5]
["a","Percentage of formal state enforcement actions that contain a compliance schedule of required actions or activities designed to return the source to compliance.  This can be in the form of injunctive relief or other complying actions.","Evaluated based on file reviews protocol using EPA or equivalent state penalty policies for setting injunctive relief.",null,null,"State-only",null,null,null,"Review Indicator"]
["b","Percentage of formal or informal enforcement responses that return sources to compliance.","Evaluated basd on file review.",null,null,"State-only",null,null,null,"Review Indicator"]
["6. Degree to which a state takes timely and appropriate enforcement actions, in accordance with policy relating to specific media.",{"$nulls":11},6]
["a","Timely action taken to address HPV sources","HPVs that were in unaddressed (no action) status for greater than 270 days. The region should evaluate how the state compares to the national average. Counts sources with at least one HPV month in the last full FY.","HPVs should be addressed with a formal action within 270 days of day zero.  Region should determine possible concern based upon state data. (HPV policy)","Sources with HPVs within fiscal year (use last full fiscal year)","State-only (with Regional-only provided)","AFS","Select the total number of flagged unaddressed HPVS (not pathways) during the FY.   Compare to the subset of those that have ten or more continuous months (>270 days) of unaddressed HPV status, starting with the first HPV month  within the FY.","IDEA","Review Indicator",null,null,"a"]
["b","Timely action to address individual HPV pathways","% of HPVs pathways that exceed the 270 day timeliness threshold. This metric looks at each individual HPV entry (6a is source-based) from the AFS \"653\" report. The region should evaluate the percentage within the state and ensure that it is in keeping with the HPV policy.","HPVs should be addressed with a formal action within 270 days of day zero (HPV determination date).  Region should determine possible concern based upon state data. (HPV policy)","HPV occurrences within fiscal year that have had at least 270 days for response","State-only (with Regional-only provided)","AFS 653 Report","Using the AFS 653 Report, select the total number of unaddressed HPV pathways during the FY.   Compare to the subset of those pathways where \"DAYS USED TO ADRS\" is >270 or \"UNADR DAYS\" is >270.   ","AFS","Review Indicator",null,null,"b"]
["c","No activity indicator - actions","No actions were taken by the state in the fiscal year.","HPV policy","All AFS","State-only","AFS","Select all state formal enforcement actions taken during the FY.  ","OTIS Management Reports","Review Indicator",null,null,"c"]
["7. Degree to which a state includes both gravity and economic benefit calculations for all penalties, appropriately using the BEN model or similar state model (where in use and consistent with national policy).",{"$nulls":11},7]
["a","Percentage of formal enforcement actions that include calculation for gravity and economic benefit consistent with applicable policies.","File review based on file review protocol using national program policy or applicable state penalty policy.",null,null,"State-only",null,null,null,"Goal"]
["8. The degree to which penalties in final enforcement actions include economic benefit and gravity in accordance with applicable penalty policies.",{"$nulls":11},8]
["a","No activity indicator - penalties","No penalties assessed by the state during the fiscal year.","HPV policy","All AFS","State-only","AFS","Select all state formal enforcement actions with penalties taken during the FY.  ","OTIS Management Reports","Review Indicator",null,null,"a"]
["b","Penalties normally included with formal enforcement actions at HPVs","Percent of enforcement actions taken at HPVs that carry any penalty. This metric does not measure appropriateness of penalties, but does flag when additional review is necessary.","Additional review necessary if state is below 80% actions against HPVs with penalty.  HPV policy assumes it is an exception that an HPV receives no penalty.","All state actions at sources that were previously HPVs","State-only","AFS","Select joint, state, or undetermined-lead HPVs in last FY with state formal enforcement actions in last FY. Compare to those with a penalty amount greater than zero.","IDEA","Review Indicator","The 80% figure is based on reality.  Guidance does not provide a set cut-off, but expects penalties unless there is an extenuating circumstance. This metric may not be useful if state has identified few HPVs.",null,"b"]
["9. Degree to which enforcement commitments in the PPA/PPG/ categorical grants (written agreements to deliver a product/project at a specified time), if they exist, are met and any products or projects are completed.",{"$nulls":11},9]
["a","State agreements (PPA/PPG/SEA, etc.) contain enforcement and compliance commitments that are met.","Review of PPAs, PPGs, SEAs, or other documents that list enforcement and compliance commitments.",null,null,"State-only",null,null,null,"Review Indicator"]
["10. Degree to which the Minimum Data Requirements are timely.",{"$nulls":11},10]
["a","Integrity of HPV data (timely entry)","Percent of HPVs that are entered to AFS more than 60 days after the HPV designation (day zero).  Measures the \"lag\" between the date of HPV designation, and the actual reporting of the HPV designation to AFS.","Timely and Appropriate Enforcement Response to High Priority Violations policy document states that the data should be entered promptly after the designation is made.  HPV entry should not be withheld until the action is completed.","All HPVs","State-only","IDEA using AFS data","IDEA compares the new HPVs that are reported into AFS each month, and compares the date that these HPVs became known to EPA to the HPV designation date that is entered into AFS. Examines the last four quarters of data.","IDEA","Review Indicator","If a problem exists in this area, it negatively impacts timeliness tracking under the HPV policy, the Watch List, award program screening, and public access. For 2005 reviews, all HPVs newly entered after Nov. 2004 are counted; eventually this will change to measuring the last four quarters of data.",null,"a"]
["b","HPV being entered in a timely manner (normally 45 days from inspection/file review)","The average number of days between the activity that triggered HPV (inspection, review, etc.) and the HPV determination.  ","HPV determinations are normally required within 45 days, however, extensions are acceptable in certain circumstances. Region should determine if state significantly departs from national averages.",null,"State-only",null,null,null,"Informational Metric/Review Indicator."]
["r","Regions should evaluate what is maintained in AFS by the State and ensure that all mimimum data required fields are properly tracked and entered according to accepted schedules, not to exceed 90 days.","Based on quarterly reporting.",{"$nulls":9},"b"]
["11. Degree to which the Minimum Data Requirements are accurate.",{"$nulls":11},11]
["a","Indicator of accurate violation/noncompliance data entry","Compares the sources with violations to the sources with HPV.  If more HPV sources appear, this is a strong indicator that violations are not fully tracked in system.","Data quality review triggered if the number of noncompliant sources is lower than the number of HPV sources, i.e., ratio > 100%. (MDRs)","Majors","Combined","AFS","Select all major CAA sources with violations within the FY.  Compare to the number of sources:   # HPVs /  # sources in non-compliance","IDEA","Data Quality","IDEA does not support state-only pull.  During file reviews, any violations noted in files should be checked against violation data in AFS.",null,"a"]
["b","Stack test results at federally-reportable sources","11b1. % of stack tests conducted and reviewed without pass/fail results code entered to AFS","0% (CMS/ICR/MDR)","All stack tests conducted at Fed-Rep Sources","State-only","AFS","Select Total Number of State Stack Tests for active majors, SM, and NESHAP minors in last FY per CAA Management Reports.  Compare to the subset of stack tests without a result code of passed or failed. ","OTIS Extended CAA Management Reports","Goal",null,null,"b"]
[null,null,"11b2. Number of sources with stack test failures at federally-reportable sources","100% of failures should be reported to AFS.","Same as above","State-only","AFS","Select the Total Number of State Stack Tests (including OWNER/OPERATOR-CONDUCTED SOURCE TEST) for active majors, SM, and NESHAP minors in last FY with a result code of failed. ","IDEA","Data Quality","Regions should ensure that states agree with the sources that failed.  Regions may want to review HPV/compliance status of these sources to ensure proper data tracking has occurred in AFS. Stack test failures often may trigger an HPV determination (see HPV policy for details)."]
["12. Degree to which the minimum data requirements are complete, unless otherwise negotiated by the region and state or prescribed by a national initiative.",{"$nulls":11},12]
[null,"For the following elements, the region should provide the counts to the state from the State Review Framework Web site and ensure the state agrees with these counts.  If there is disagreement, additional evaluation should be conducted to determine the source of the discrepancy and to develop an action plan for making appropriate corrections."]
["a","Title V universe","12a1. AFS operating majors","Region and state should agree on universe count.",null,"N/A","AFS","Select all operating major sources from AFS.","IDEA","Data Quality",null,null,"a"]
[null,null,"12a2. AFS operating majors w/ air program code = V","Region and state should agree on universe count.",null,"N/A","AFS","Select all operating major sources from AFS that have an Air Program Code for Title V (APC1 = V).","IDEA","Data Quality","The region and state should coordinate with Title V permit personnel to ensure accurate Title V universe counts.  For TOPS, managed by OAQPS, see http://ttnwww.rtpnc.epa.gov/tops/home.cfm (for EPA Intranet users)."]
["b","State agrees with source count for Major, SM, NESHAP minor sources","12b1. Major","Minimum data requirements are listed in the AFS Information Collection Request.",null,"N/A","AFS","Select all active major sources from AFS.","OTIS Extended CAA Management Reports","Data Quality",null,null,"b"]
[null,null,"12b2. Synthetic minor","Minimum data requirements are listed in the AFS Information Collection Request.",null,"N/A","AFS","Select all active SM sources from AFS.","OTIS Extended CAA Management Reports","Data Quality"]
[null,null,"12b3. NESHAP minor","Minimum data requirements are listed in the AFS Information Collection Request.",null,"N/A","AFS","Select all active NESHAP minor sources from AFS.","IDEA","Data Quality"]
["c","Subprogram universe is accurate in AFS (NSPS, NESHAP and MACT).","12c1. Subpart designation per CAA - NSPS",null,null,"N/A","AFS","Select all active, federally reportable NSPS sources from AFS based on their air program codes.","IDEA","Data Quality","Note: See subpart implementation schedule in the ICR.",null,"c"]
[null,null,"12c2. Subpart designation per CAA - NESHAP",null,null,"N/A","AFS","Select all active, federally reportable NESHAP sources from AFS based on their air program codes.","IDEA","Data Quality"]
[null,null,"12c3. Subpart designation per CAA - MACT",null,null,"N/A","AFS","Select all active, federally reportable MACT sources from AFS based on their air program codes.","IDEA","Data Quality"]
["d","Compliance monitoring counts complete","12d1 - Sources with FCEs in reporting period","Minimum data requirements are listed in the AFS Information Collection Request.","Operating  Major, SM, and NESHAP Minor sources","State-only","AFS","Select state, relevant FY, and Full Compliance Evaluation Coverage (state only).","OTIS Extended CAA Management Reports","Data Quality","Operating sources are those with OPST=O,T,I",null,"d"]
[null,null,"12d2 - Total FCEs completed in reporting period","Minimum data requirements are listed in the AFS Information Collection Request.","All Major, SM, and NESHAP Minor sources","State-only","AFS","Select state, relevant FY, and Total Number of State Full Compliance Evaluations (FCEs) at RECAP Universes (state only).","OTIS Extended CAA Management Reports","Data Quality","All sources are those with OPST=O,T,I,P,C,X"]
[null,null,"12d3 - Number of PCEs reported to AFS in reporting period","State PCEs are not required (unless negotiated) but will be included for informational purposes when reported.","All sources","State-only","AFS",null,"IDEA","Informational-only"]
["e","Historical non-compliance counts complete","Number of sources that had violations at any point during the reporting period","Minimum data requirements are listed in the AFS Information Collection Request.","all","Combined","AFS","Select CAA, states by Region, the relevant FY, and sources in violation","OTIS Management Reports","Data Quality",null,null,"e"]
["f","Notice of violation counts complete","12f1 - Number of NOVs issued in last FY","Minimum data requirements are listed in the AFS Information Collection Request.","all","State-only","AFS","For total NOVs, select CAA, states by Region, the relevant FY, and state notices of violation.  ","OTIS Management Reports","Data Quality","During the review, Regions may want to compare sources with NOVs to sources with HPV listings to assist in determining integrity of HPV data.",null,"f"]
[null,null,"12f2 - Number of sources with NOVs in last FY","Minimum data requirements are listed in the AFS Information Collection Request.","all","State-only","AFS","For sources with NOVs, use OTIS CAA query.","OTIS CAA query","Data Quality","During the review, Regions may want to compare sources with NOVs to sources with HPV listings to assist in determining integrity of HPV data."]
["g","HPV counts complete","12g1 - Number of new HPVs (pathways) in last FY","Minimum data requirements are listed in the AFS Information Collection Request.","all","State-only","AFS","Select CAA, states by Region, the relevant FY, and # new State HPVs","IDEA","Data Quality",null,null,"g"]
[null,null,"12g2 - Number of sources in HPV in last FY","Minimum data requirements are listed in the AFS Information Collection Request.","all","State-only","AFS","In OTIS MRs, select CAA, states by Region, the relevant FY, and # sources with new HPV (for a combined number only).","IDEA","Data Quality"]
["h","Formal action counts complete","12h1 - Number of actions issued in last FY","Minimum data requirements are listed in the AFS Information Collection Request.","all","State-only","AFS","For total actions, select CAA, states by Region, the relevant FY, and state total formal actions.","OTIS Management Reports","Data Quality",null,null,"h"]
[null,null,"12h2 - Number of sources with actions in last FY","Minimum data requirements are listed in the AFS Information Collection Request.","all","State-only","AFS","For source count, select CAA, states by Region, the relevant FY, and # sources with state actions.  ","OTIS Management Reports","Data Quality"]
["i","Assessed penalties complete","Total dollar amount of assessed penalties in last FY","Minimum data requirements are listed in the AFS Information Collection Request.","all","State-only","AFS","Select CAA, states by Region, the relevant FY, and total state penalties","OTIS Management Reports","Data Quality",null,null,"i"]
["j","Number of major sources missing CMS policy applicability","Identifies major sources in AFS that have not been identified with CMS applicability.  Without CMS categories and frequencies, these sources are not included in the automatic unknown compliance status generation.","The CMS policy indicates that all major sources be evaluated within a specified timeframe as negotiated with the state and region.","Active majors","Combined","AFS/ IDEA","Select all active major sources from AFS where the CMS Source Category (CMSC) field is blank.","IDEA","Review Indicator",null,null,"j"]

{"sheet":"CWA metrics","index":1,"n_rows":75,"n_cols":13,"rows_kept":75,"cut":false,"hidden":false,"merged":[[71,8,72,8],[71,9,72,9],[71,6,72,6],[73,1,74,1],[73,6,74,6],[73,10,74,10],[73,5,74,5],[73,8,74,8],[73,9,74,9],[50,12,53,12],[73,0,74,0],[73,3,74,3],[73,4,74,4],[71,5,72,5],[71,0,72,0],[71,3,72,3],[71,4,72,4],[71,1,72,1],[71,10,72,10],[24,0,24,11],[21,2,21,10],[26,0,26,11],[32,0,32,11],[31,2,31,9],[27,0,28,0],[27,1,28,1],[27,3,28,3],[29,1,30,1],[29,0,30,0],[53,2,53,10],[57,0,58,0],[59,1,60,1],[62,1,63,1],[50,1,53,1],[50,0,53,0],[54,0,56,0],[54,1,56,1],[69,10,70,10],[69,0,70,0],[69,4,70,4],[69,6,70,6],[69,3,70,3],[69,5,70,5],[64,0,66,0],[64,3,66,3],[69,1,70,1],[57,1,58,1],[64,1,66,1],[67,0,68,0],[62,0,63,0],[59,0,60,0],[67,1,68,1],[1,0,1,1],[2,0,6,1],[18,0,19,0],[18,3,19,3],[18,1,19,1],[15,0,15,10],[13,0,13,1],[16,0,16,11],[13,2,13,12],[22,0,22,11],[7,0,7,1],[8,0,12,1],[7,2,7,12],[8,2,8,12],[20,2,20,3],[9,2,9,12],[10,2,10,12],[11,2,11,12],[12,2,12,12],[45,0,45,11],[47,0,47,11],[49,0,49,11],[18,12,19,12],[27,12,28,12],[29,12,30,12],[35,0,35,11],[38,0,38,11],[40,0,40,11],[43,0,43,11],[67,12,68,12],[69,12,70,12],[71,12,72,12],[54,12,56,12],[57,12,58,12],[59,12,60,12],[62,12,63,12],[73,12,74,12],[14,11,15,11],[0,0,0,12],[1,2,1,12],[2,2,2,12],[3,2,3,12],[4,2,4,12],[5,2,5,12],[6,2,6,12],[64,12,66,12]],"flags":["formulas_cached_only"]}
["CWA Metrics - Please answer the following three questions."]
["1.",null,"What new metrics are needed? (Please also explain how the proposed metric would support your program evaluation.)"]
[null,null,"1.  "]
[null,null,"2.  "]
[null,null,"3.  "]
[null,null,"4.  "]
[null,null,"5.  "]
["2.",null,"Which metrics should we consider removing? (Please provide a brief justification.)"]
[null,null,"1.  "]
[null,null,"2.  "]
[null,null,"3.  "]
[null,null,"4.  "]
[null,null,"5.  "]
["3.",null,"Do you have any suggestions for improving the existing metrics?  (Please list suggested improvements in the far right column of the table below (\"Your Suggestions\").  Please type each of your suggestions in the yellow shaded region next to the corresponding metric.)"]
[null,"Name","Description","Guidance Requirement or Goal","Universe","Combined Region/State or State-only Data?","Original Source of Data Provided by EPA","Selection Criteria Explanation","Data Pulled From","Metric Type","Notes","Your Suggestions         ."]
["NOTE: Due to changing WENDB requirements, this table will need to be updated when data for new facility universes are available.  Some measures will need to change as a result of ICIS-NPDES and the Wet Weather SNC Policy that is under development.  "]
["1. Degree to which state program has completed the universe of planned inspections (addressing core requirements and federal, state, and regional priorities).",{"$nulls":11},1]
["a","Inspection coverage - NPDES majors","EPA has a goal of 100% annual inspection coverage at NPDES majors.","100%.  If the region has negotiated a different percent with the state, the region should include the goal under 1r.","Active NPDES majors with individual and general permits","State-only (but separate combined % also provided)","PCS/ICIS-NPDES","Select active NPDES major permits as the universe.  Count facilities that have received one or more state or EPA inspections within the reporting year.  Reconnaissance inspections are not counted at primary industries or at approved pretreatment facilities.","OTIS State Review Framework Data Metrics Tool","Goal","Corresponds to the PIP National Management Report inspection coverage measure for major sources.  Note re: universe counts: OECA researching option of capturing End-of-Year universe starting with FY2005 for use where universe counts are required.  Also note that all references to fiscal year are federal; discrepancies between state and federal fiscal years may cause discrepancies in EPA and state numbers. Should a region/state want to look at a different time frame, use 1r below.",null,"a"]
["b","Inspection coverage - NPDES non-majors","Inspections at NPDES non-majors with individual permits.","This metric is for information purposes only.  It provides inspection coverage for NPDES non-majors with effluent limits and DMRs. In some cases, states may agree to inspection percentages for non-majors in PPAs, PPGs, or may be meeting basic goals by \"trading\" major for non-major inspections.  This provides information to the region.","Active non-majors with individual permits","State-only (but separate combined % also provided)","PCS/ICIS-NPDES","Select active non-majors with individual permits as the universe. Count facilities that have received one or more state or EPA inspections within the year. (PCS metrics measure inspections during the NPDES Review Year, while ICIS-NPDES metrics measure the federal fiscal year. OECA will update the PCS logic after the ICIS-NPDES interface is completed.) Reconnaissance inspections are not counted at primary industries or at approved pretreatment facilities.","OTIS State Review Framework Data Metrics Tool","Informational-only","Corresponds to PIP National Management Report inspection coverage measure for non-majors.",null,"b"]
[null,null,"Inspections at NPDES non-majors with general permits.",null,"Active non-majors with general permits","State-only (but separate combined % also provided)","PCS/ICIS-NPDES","Select active non-majors with general permits as the universe. Count facilities that have received one or more state or EPA inspections within the year. Reconnaissance inspections are not counted at primary industries or at approved pretreatment facilities.  ","OTIS State Review Framework Data Metrics Tool","Informational-only"]
["c","Other inspections performed (beyond facilities indicated in 1a and 1b) (Coverage)","This metric is for information purposes only.  It provides inspection coverage for NPDES permittees that do not have effluent limits and DMRs. In some cases, states may agree to inspection percentages for non-majors in PPAs, PPGs, or may be meeting basic goals by \"trading\" major for non-major inspections.  This provides information to the region.",null,"Active facilities without effluent limits and that do not regularly submit DMRs","State-only (but separate combined % also provided)","PCS/ICIS-NPDES","Select active major and non-major permits without effluent limits and that do not regularly submit DMRs (non-standard, non-general permits) as the universe.  Count facilities that have received one or more state or EPA inspections within the reporting year.  Reconnaissance inspections are not counted at primary industries or at approved pretreatment facilities.","IDEA","Informational-only","May be a precursor for eventual measurement of inspections at wet weather inspections when these universes are cleaned up in the database.",null,"c"]
["r","Regions can track yearly commitments or multi-year plans.  Reserved for inspection plan targets negotiated between the region and state in PPAs and grant agreements.","Focus on the negotiated percent of non-majors and regional tracking of any \"major for non-major\" inspection trades.  Also, discrepancies between state and federal fiscal years may cause discrepancies between EPA and state numbers.",{"$nulls":9},"r"]
["2. Degree to which inspection reports and compliance reviews document inspection findings, including accurate description of what was observed to sufficiently identify violation.",{"$nulls":11},2]
["a","Percentage of inspection reports that are adequately documented.","Evaluate based on file review.  File Review Protocol should look closely at evaluating the processes used by states to implement the Violation Review Action Criteria (VRAC) which provides guidance on how Regions and delegated states should review all violations for potential enforcement response.",null,null,"State-only",null,null,null,"Review Indicator"]
["3. Degree to which inspection reports are completed in a timely manner, including timely identification of violations.",{"$nulls":11},3]
["a","Percentage of Inspection Reports which identify potential violations in the file within a given time frame established by the Region and state.","Review should also identify which state inspections identified violations, and determine whether those violations were entered into PCS (single-event violations).",null,null,"State-only",null,null,null,"Review Indicator"]
["4. Degree to which significant violations (e.g., significant noncompliance and high priority violations) and supporting information are accurately identified and reported to EPA national databases in a timely manner. ",{"$nulls":11},4]
["a","Single-event violations reported to national system (non-automated violations arising from inspections and compliance monitoring)","4a1. Single-event violations at majors. This measure assesses whether violations determined by means other than automated discharge to limits comparisons are being reported and tracked in PCS.","States should be entering violations arising from major facility compliance monitoring.  Violations found at non-major facilities are not yet required to be reported, but are included as a separate measure as a placeholder.  Review of how violation determinations from inspections or other means are being reported is required for regions/states that show zero or very few single-event violations in PCS.","Active majors","Combined (IDEA does not provide ability to separate EPA from state)","PCS/ICIS-NPDES","Select all major facilities with single-event violations identified within the fiscal year under review.","OTIS State Review Framework Data Metrics Tool","Review Indicator","Several regions and states requested more detailed guidance on how to enter and track single-event violations.  OECA clarified this in a data entry guide released in September 2005 (Interim Single Event Violation Data Entry Guide for PCS).  If no activity is listed at all, regions should discuss the state's capability to send this information into the database.",null,"a"]
[null,null,"4a2. Single-event violations at non-majors. This measure assesses whether violations determined by means other than DMRs are being reported and tracked in PCS. ",null,"Active non-majors","Combined (IDEA does not provide ability to separate EPA from state)","PCS/ICIS-NPDES","Select all non-major facilities with single-event violations identified within the fiscal year under review.","OTIS State Review Framework Data Metrics Tool","Informational-only","This measure will eventually change to a review indicator after data requirements are in effect for a full year."]
["b","Frequency of SNC","4b1. Facilities in SNC during reporting year","The list of SNC facilities is accurate only if the permit limits with their DMRs and permit schedules and compliance schedules with their report dates are routinely entered into the national system. Such data are required. Minimum data requirements are listed in the 1985 PCS Policy Statement.","Majors","Combined (PCS does not provide ability to separate EPA from state)","PCS/ICIS-NPDES","Select and count the total number of NPDES majors in SNC during last full FY.Select the number of NPDES major facilities that have had at least one occurrence of D, E, S, T, or X in the last year.","IDEA","Review Indicator","Region can examine data in element 12 to help support this analysis.",null,"b"]
[null,null,"4b2. SNC Rate","Percent of major facilities in SNC. The list of SNC facilities is accurate only if the permit limits with their DMRs and permit schedules and compliance schedules with their report dates are routinely entered into the national system. Such data are required.","Majors","Combined (PCS does not provide ability to separate EPA from state)","PCS/IDEA/ICIS-NPDES","Percent of major facilities in SNC; select the number of facilities that have had at least one occurrence of D, E, S, T, or X in the last year and divide by total major facilities.","IDEA","Review Indicator","Region can examine data in element 12 to help support this analysis."]
["c","Wet Weather SNC placeholder","Metric(s) to be developed in the future.",{"$nulls":9},"c"]
["5. The degree to which state enforcement actions include required corrective or complying actions (injunctive relief) that will return facilities to compliance in a specific time frame.",{"$nulls":11},5]
["a","Percentage of formal state enforcement actions that contain a compliance schedule of required actions or activities designed to return the source to compliance.  This can be in the form of injunctive relief or other complying actions.","Evaluated based on file review protocol using EPA or equivalent state penalty policy.",null,null,"State-only",null,null,null,"Review Indicator"]
["b","Percentage of actions or responses other than formal enforcement that return source to compliance.","Evaluated based on file review protocol using EPA or equivalent state penalty policy.",null,null,"State-only",null,null,null,"Review Indicator"]
["6. Degree to which a state takes timely and appropriate enforcement actions, in accordance with policy relating to specific media.",{"$nulls":11},6]
["a","Timely action taken to address SNC","Guidance calls for enforcement action before two quarters of QNCR effluent violation at same pipe, same parameter.","Number of facilities without timely action does not exceed 2% of active major universe throughout the fiscal year, based on the QNCR Guidance Manual.","Majors","Combined","PCS/ICIS-NPDES","Sum all facilities that meet the criteria of consecutive SNC effluent violations with no action and other unresolved SNC violations in consecutive quarters: QNCR Codes E, X, or S for 2 or more consecutive quarters and SNC effluent violations of the same pipe and parameter for 2 or more consecutive quarters, and no formal  enforcement action within one year from the 2nd consecutive QNCR quarter; OR three consecutive quarters of SNC-level effluent violations at the same pipe and same parameter, and no formal enforcement action within one year from the 3rd consecutive violation quarter; OR QNCR Codes E, X, or S for 2 or more consecutive quarters and  2 or more consecutive, unresolved (PCS RNC resolution code = 1 or A) SNC effluent violations at the same pipe and of the same parameter; OR QNCR Code = D (DMR Nonreceipt) for 2 or more consecutive quarters and no formal enforcement action within one year from the 2nd consecutive QNCR quarter; OR, QNCR Code = S or T (Compliance Schedule Violations) for 2 or more consecutive quarters and at least one current open compliance schedule violation. Current refers to the date of the data pull. Calculate whether this number is greater than 2% of the current major universe.","OTIS State Review Framework Data Metrics Tool","Goal","Though other media use a state-only metric, this is appropriately combined since DMRs can be addressed by EPA or state.  Hard to separate data.  Results also will show what 2% of the current major universe is.",null,"a"]
["b","No activity indicator (actions)","No actions taken by state in fiscal year.",null,"All facilities","State-only","PCS/ICIS-NPDES","Select OTIS CWA Management Reports query: State Enforcement - Total Formal Actions for last full FY.","OTIS Management Reports","Review Indicator",null,null,"b"]
["7. Degree to which a state includes both gravity and economic benefit calculations for all penalties, using the BEN model or similar state model (where in use and consistent with national policy).",{"$nulls":11},7]
["a","Percentage of formal enforcement actions that include calculation for gravity and economic benefit consistent with applicable policies.","File review based on file review protocol using national program policy and the BEN or equivalent models.",null,null,"State-only",null,null,null,"Goal"]
["8. The degree to which penalties in final enforcement actions include economic benefit and gravity in accordance with applicable penalty policies.",{"$nulls":11},8]
["a","No activity indicator (penalties) **","No penalties taken by state in fiscal year.  If state is not entering penalties to PCS, regions should request information.","Data not required, but expected to be when ICIS-NPDES requirements are final. ","All facilities","State-only","PCS/ICIS-NPDES","Select OTIS CWA Management Reports query: Penalties - Total State Penalties for last full FY.","OTIS Management Reports","Informational-only; will convert to review indicator once ICIS-NPDES Policy Statement is finalized.","**Penalty data not currently required to be reported to PCS/ ICIS-NPDES. Because penalty data provide important information about the health of the compliance and enforcement program, region may need to seek information from the state regarding penalty amounts - particularly during the file reviews. This indicator can be based on state-provided data until ICIS-NPDES requirements are final.  If preliminary data shows \"0\" for this metric, states may replace PCS/ ICIS-NPDES data with their own data.",null,"a"]
["b","Penalties normally included with formal enforcement actions","Percent of enforcement actions that carry any penalty. This metric does not measure appropriateness of penalties, but does flag when additional review is necessary.","Data not required, but expected to be when ICIS-NPDES requirements are final. ","All facilities with state actions","State-only","PCS/ICIS-NPDES","Select OTIS CWA Management Reports query: State Enforcement - % State Actions With Penalty for last full FY.","OTIS Management Reports","Informational-only; will convert to review indicator once ICIS-NPDES Policy Statement is finalized.","See Notes for 8a.  If state is not entering penalties into PCS, region should request information from state. (There may be legitimate circumstances for facilities to make this list.)",null,"b"]
["9. Degree to which enforcement commitments in the PPA/PPG/ categorical grants (written agreements to deliver a product/project at a specified time) are met and any products or projects are completed.",{"$nulls":11},9]
["a","State agreements (PPA/PPG/SEA, etc.) contain enforcement and compliance commitments that are met.","Review of PPAs, PPGs, SEAs, or other documents that list enforcement and compliance commitments.",null,null,"State-only",null,null,null,"Review Indicator"]
["10. Degree to which the Minimum Data Requirements are timely.",{"$nulls":11},10]
["a","Regions should evaluate what is maintained in PCS by the State and ensure that all minimum data elements are properly tracked and entered according to accepted schedules.","File review to assess timely entry of data into PCS.",null,null,"State-only",null,null,null,"Review Indicator"]
["11. Degree to which Minimum Data Requirements are accurate.",{"$nulls":11},11]
["a","Actions are linked to the violations they address","Number of enforcement actions with EVTP (enforcement violation type) codes entered.","The region should ensure that all enforcement actions are properly linked to violations noted in PCS.  If a state is below 80%, the region should perform additional review of data quality.","All facilities with actions","State-only","PCS/ICIS-NPDES","Select all state formal enforcement actions within the FY.  Compare to the subset of these that have an associated EVTP (enforcement violation type) code(s) entered. (Note: \"ERFN restriction\" [see metric 12h2 for explanation] is not applied to this metric.)","IDEA","Data Quality","Entry of these links helps create an accurate Watch List. Region can enter online data errors via OTIS for inaccurate data.",null,"a"]
["12. Degree to which the minimum data requirements are complete, unless otherwise negotiated by the region and state or prescribed by a national initiative.",{"$nulls":11},12]
["a","Active facility universe counts accurate for all NPDES permit types","12a1. NPDES majors with individual and general permits","Region and state should agree on active universe.",null,"State-only","PCS/ICIS-NPDES","Select and count the total number of  NPDES active major dischargers. ","OTIS Management Reports or OTIS CWA query","Data Quality","Required data",null,"a"]
[null,null,"12a2. NPDES non-majors with individual permits","Region and state should agree on active universe.",null,"State-only","PCS/ICIS-NPDES","Select and count the total number of  NPDES active non-major dischargers with individual permits.","OTIS Management Reports or OTIS CWA query","Data Quality","Required data"]
[null,null,"12a3. NPDES non-majors with general permits","Region and state should agree on active universe.",null,"State-only ","PCS/ICIS-NPDES","Select and count the total number of  NPDES active non-major dischargers with general permits.","OTIS Management Reports or OTIS CWA query","Data Quality","Required data"]
[null,null,"When data are required, permit components should be tracked here including CSO, CAFO, Pretreatment, POTW, Biosolids, Stormwater (small/large MS4, construction, non-construction).  "]
["b","Majors permit limits and DMR entry complete","12b1. Limits at majors   ","95% entry standard for majors","Active majors","Combined","PCS/ICIS-NPDES","Active major individual and general permits which have at least one correctly coded limit in PCS, where: the permit expiration date and at least one limit end date are after the review date, and the start date(s) for the same limit(s) is/are before the review date, OR where: the permit is expired or administratively continued (while the permit application is under review by the state) (permit expiration date is before the review date), and at least one limit end date is equal to or later than the permit expiration date.","IDEA","Goal","100% data entry ensures accurate SNC calculation and accurate ECHO reports.  Required data.  This metric evaluates data as a snapshot in time, based on the last day +1 of the FY for prior FYs, or the last day +1 of the current official quarter for the current FY. (For example, FY2004 is based upon the status as of 10-01-2004.)",null,"b"]
[null,null,"12b2. DMR entry rate at majors. The number of  DMR forms received from major dischargers and entered into PCS for the last quarter of the FY divided by the number of DMR forms expected for majors in that quarter.","95% entry standard for majors","Active majors","Combined","PCS/ICIS-NPDES","The number of  DMR forms received from major dischargers (municipal+nonmunicipal) and entered into PCS for the most recent quarter divided by the number of DMR forms expected for majors in that quarter.","PCS","Goal","100% data entry ensures accurate SNC calculation and accurate ECHO reports.  Required data.  Data for the current FY will be the most recent completed quarter, as noted on the Results page. DMR Forecasting Submission Date- Both (SUDB) is lagged one month from the quarter start and end dates."]
[null,null,"12b3. Rate of manual override of SNC to a compliant status",null,"Active majors","Combined","PCS/ICIS-NPDES","Rate of manual override of SNC to a compliant status in the last full FY. The denominator counts facilities that are PCS active majors with a QNCR value of D, E, S, T, or X (SNC values) within the FY. The numerator is a subset of the denominator where the SNC value was manually changed within the same quarter to a value of P, R, or C.","IDEA","Data Quality","There are legitimate circumstances for manual overrides; however, when done excessively, activity should be reviewed."]
["c","Non-majors permit limits and DMR entry","12c1. DMR limit entry rate","This is not currently a required field - no review needed - information only.","Active non-majors","Combined","PCS/ICIS-NPDES","Active non-major individual and general permits which have at least one correctly coded limit in PCS, where: the permit expiration date and at least one limit end date are after the review date, and the start date(s) for the same limit(s) is/are before the review date, OR where: the permit is expired or administratively continued (while the permit application is under review by the state) (permit expiration date is before the review date), and at least one limit end date is equal to or later than the permit expiration date.","IDEA","Informational-only","Pending new data requirements.  Information not required now.  The information can be helpful in verifying Annual Quarterly Noncompliance Report and tracking progress toward meeting new data entry requirements (will help region/state understand workload challenges for non-major DMR entry).  This metric evaluates data as a snapshot in time, based on the last day +1 of the FY for prior FYs, or the last day +1 of the current official quarter for the current FY.  (For example, FY2004 is based upon the status as of 10-01-2004.)",null,"c"]
[null,null,"12c2. DMR entry rate.  The number of  DMR forms received from non-major dischargers and entered into PCS for the last quarter of the FY divided by the number of DMR forms expected for non-majors in that quarter.","This is not currently a required field - no review needed - information only.","Active non-majors","Combined","PCS/ICIS-NPDES","The number of  DMR forms received from non-major dischargers (municipal+nonmunicipal) and entered into PCS for the most recent quarter divided by the number of DMR forms expected for non-majors in that quarter.","IDEA","Informational-only","Pending new data requirements.  Information not required now.  The information can be helpful in verifying Annual Quarterly Noncompliance Report.  Data for the current FY will be the most recent completed quarter, as noted on the Results page.  DMR Forecasting Submission Date- Both (SUDB) is lagged one month from the quarter start and end dates."]
["d","Inspection counts complete","12d1. Number of facilities inspected in last FY","Minimum data requirements are listed in the 1985 PCS Policy Statement.","Active ","State-only (with region-only provided)","PCS/ICIS-NPDES","Select and count the total number of state NPDES inspections at any permitee during last full FY.","OTIS Management Reports","Data Quality","Required data",null,"d"]
[null,null,"12d2. Total number of inspections performed in last FY","Minimum data requirements are listed in the 1985 PCS Policy Statement.","Active ","State-only (with region-only provided)","PCS/ICIS-NPDES","Select and count the total number of state NPDES inspections at any permitee during last full FY.","OTIS Management Reports","Data Quality","Required data"]
["e","Percent of violations linked to activity that identified the violation","To be developed.  Will be discussed in more detail in upcoming single event violation data entry guide.",null,"Active ","State-only (with region-only provided)","ICIS-NPDES",null,"IDEA",null,"Placeholder only at this time.  Data not required before ICIS-NPDES.",null,"e"]
["f","Notice of violation counts complete","12f1. Number of facilities with state NOVs in last FY","Minimum data requirements are listed in the 1985 PCS Policy Statement.","Active ","State-only (with region-only provided)","PCS/ICIS-NPDES","For facilities with NOVs, use OTIS CWA query.","OTIS CWA query","Data Quality","Required data for major facilities. Output should be sortable by major/non-major.",null,"f"]
[null,null,"12f2. Total number of state NOVs issued in last FY","Minimum data requirements are listed in the 1985 PCS Policy Statement.","Active ","State-only (with region-only provided)","PCS/ICIS-NPDES","For total NOVs, select CWA, states by Region, the relevant FY, and state notices of violation.  ","OTIS Management Reports","Data Quality","Required data for major facilities. Output should be sortable by major/non-major."]
["g","Quality of violation data at non-major facilities (that regularly submit DMRs)","12g1. Noncompliance rate in database at non-major facilities in last fiscal year","Data are not required for 12g1 and 12g3; however, this metric provides a general sense of the quality of the data in the system (particularly given these data are shown in ECHO).  This measure will become more important when the complete set of minors limits and DMRs are reported.","Non-majors (that regularly submit DMRs)","Combined","PCS/ICIS-NPDES","Select all active minor facilities with standard permit types.  Compare to the subset of these that have at least one QNCR violation within the FY.","IDEA","Informational-only","Data are not required for 12g1; however, this metric provides a general sense of the quality of the data in the system before ICIS-NPDES is functional.",null,"g"]
[null,null,"12g2. Noncompliance rate reported to EPA under the Annual Noncompliance Report (ANCR)",null,"Non-majors (that regularly submit DMRs)","Combined","ANCR","manual count","ANCR","Informational-only","While most data metrics show data for the last full fiscal year, for this metric, the data metrics Web site displays the last full calendar year that is available.  Historically it has taken eleven to fifteen months after the end of a calendar year to complete the ANCR for that year.  "]
[null,null,"12g3. Number of facilities in database with DMR non-receipt for three continuous years",null,"Non-majors (that regularly submit DMRs)","Combined","PCS/ICIS-NPDES","Select all active minor facilities with standard permit types that have had a QNCR violation value \"D\" (DMR non-receipt) in every quarter for the past three years, starting with the most recent official data reported by PCS to IDEA.","IDEA","Informational-only","Data are not required for 12g3; however, this metric provides a general sense of the quality of the data in the system before ICIS-NPDES is functional."]
["h","Formal action counts complete","12h1. Facilities with formal actions","Minimum data requirements are listed in the 1985 PCS Policy Statement.","All facilities ","State-only","PCS/ICIS-NPDES","Select and count any state formal enforcement action in last full FY.","OTIS Management Reports","Data Quality","Required data for major facilities. Output should be sortable by major/non-major.",null,"h"]
[null,null,"12h2. Total formal actions taken","Minimum data requirements are listed in the 1985 PCS Policy Statement.","All facilities ","State-only","PCS/ICIS-NPDES","For total actions, select CWA, states by Region, the relevant FY, and state total formal actions.  (Note: The \"ERFN restriction\" is applied to this metric (ERFN = Enforcement Action File Number).  When counting state formal enforcement actions (EAs) and/or penalties, if more than one EA (per RECAP) carries the same Case File/Docket number (i.e., are the result of the same enforcement response), then only one EA may be counted within the fiscal year in which they occurred, even if the EAs are recorded at different facilities and/or on different dates.)","OTIS Management Reports","Data Quality","Required data for major facilities. Output should be sortable by major/non-major."]
["i","Assessed penalties complete *(Data are not currently required from states)","12i1. Formal actions with penalties","This is not currently a required field - no review needed - information only.","All facilities ","State-only","PCS/ICIS-NPDES","Select and count any state formal enforcement action that includes a penalty in last full FY.","OTIS Management Reports","Informational-only","Pending new data requirements.  Also see 8a Notes. State may replace data.",null,"i"]
[null,null,"12i2. Total state penalties",{"$nulls":4},"Select and sum the penalty value for any state formal enforcement action that includes a penalty in last full FY.","OTIS Management Reports","Informational-only"]
["j","Major facilities with compliance schedule violations","12j1. Permittees with unresolved compliance schedule violations, as of the end of the fiscal year.","Compliance schedules are WENDB for major permits.  Review of violation determinations is recommended for regions/states that show zero or very few compliance schedule violations in PCS based upon a current snapshot.","Active majors","Combined","PCS/ICIS-NPDES","Select all active, major permittees with unresolved, detected compliance schedule violations as of the end of the fiscal year, based on the compliance schedule date (DTSC).","IDEA","Data Quality","This measure assesses whether compliance schedule violations are being reported and tracked in PCS. Compliance schedule events and violations from prior fiscal years may be counted here if they meet all of the selection criteria. Compliance schedule violations at non-major facilities are not required until the ICIS-NPDES Policy Statement is final, so will not be measured until after that.",null,"j"]
[null,null,"12j2. Permittees with compliance schedule events with a compliance schedule date prior or equal to the end of the fiscal year.",{"$nulls":4},"Select all active, major permittees with compliance schedule events, with a compliance schedule date (DTSC) prior or equal to the end of the fiscal year."]
["k","Major facilities with permit schedule violations","12k1. Permittees with unresolved permit schedule violations, as of the end of the fiscal year.","Permit schedules are WENDB for major permits.  Review of violation determinations is recommended for regions/states that show zero or very few permit schedule violations in PCS based upon a current snapshot. (In PCS Permit Schedules are called Compliance Schedules, the same as those from EAs.)","Active majors","Combined","PCS/ICIS-NPDES","Select all active, major permittees with unresolved, detected permit schedule violations as of the end of the fiscal year, based on the compliance schedule date (DTSC).","IDEA","Data Quality","This measure assesses whether compliance schedule violations are being reported and tracked in PCS. Permit schedule events and violations from prior fiscal years may be counted here if they meet all of the selection criteria. ",null,"k"]
[null,null,"12k2. Permittees with permit schedule events with a schedule date prior or equal to the end of the fiscal year.",{"$nulls":4},"Select all active, major permittees with permit schedule events, with a compliance schedule date (DTSC) prior or equal to the end of the fiscal year."]

{"sheet":"RCRA metrics","index":2,"n_rows":68,"n_cols":14,"rows_kept":68,"cut":false,"hidden":false,"merged":[[13,2,13,13],[13,0,13,1],[1,2,1,13],[8,0,12,1],[8,2,8,13],[9,2,9,13],[10,2,10,13],[11,2,11,13],[12,2,12,13],[63,0,64,0],[63,1,64,1],[54,1,57,1],[45,0,45,12],[53,1,53,11],[49,0,50,0],[49,1,50,1],[52,0,52,12],[26,0,26,12],[31,0,31,12],[65,0,66,0],[65,1,66,1],[61,0,62,0],[61,1,62,1],[58,0,59,0],[58,1,59,1],[54,0,57,0],[48,0,48,12],[21,2,21,10],[15,0,15,12],[22,0,22,12],[24,0,24,12],[61,13,62,13],[63,13,64,13],[65,13,66,13],[41,13,42,13],[49,13,50,13],[54,13,57,13],[58,13,59,13],[0,0,0,13],[1,0,1,1],[4,2,4,13],[7,0,7,1],[2,0,6,1],[5,2,5,13],[6,2,6,13],[7,2,7,13],[2,2,2,13],[3,2,3,13],[34,0,34,12],[37,0,37,12],[39,0,39,12],[43,0,43,12],[41,0,42,0],[41,1,42,1],[41,11,42,11]],"flags":["formulas_cached_only"]}
["RCRA Metrics - Please answer the following three questions."]
["1.",null,"What new metrics are needed? (Please also explain how the proposed metric would support your program evaluation.)"]
[null,null,"1.  "]
[null,null,"2.  "]
[null,null,"3.  "]
[null,null,"4.  "]
[null,null,"5.  "]
["2.",null,"Which metrics should we consider removing? (Please provide a brief justification.)"]
[null,null,"1.  "]
[null,null,"2.  "]
[null,null,"3.  "]
[null,null,"4.  "]
[null,null,"5.  "]
["3.",null,"Do you have any suggestions for improving the existing metrics?  (Please list suggested improvements in the far right column of the table below (\"Your Suggestions\").  Please type each of your suggestions in the yellow shaded region next to the corresponding metric.)"]
[null,"Name","Description","Guidance Requirement or Goal","Universe","Combined Region/State or State-only Data?","Original Source of Data Provided by EPA","Selection Criteria Explanation ","Data Can Be Pulled From","Guidance","Metric Type","Notes","Your Suggestions"]
["1. Degree to which state program has completed the universe of planned inspections (addressing core requirements and federal, state, and regional priorities).",{"$nulls":12},1]
["a","Inspection coverage for operating Treatment, Storage, and Disposal Facilities (TSDFs)","Per RCRA, region/state should inspect all operating TSDFs within two years; however, if facilities included on the list were not operating for the full two years, inspection is not expected.","Operating TSDFs should be inspected every 2 years.","Operating TSDFs","State-only (but separate combined % also provided)","RCRAInfo","Select all operating TSDFs in RCRAInfo that have been inspected during the past two FYs.   Count each facility once regardless of inspection frequency within the timeframe.","IDEA","NPG","Goal","Note re: universe counts: OECA saving End-of-Year universe starting with FY2005 for use where universe counts are required; researching possible enhancements to allow for this on Web site. Trade agreements that are negotiated under national guidance should be tracked using metric 1r below. ",null,"a"]
["b","Annual inspection coverage - Active Large Quantity Generators","National measure guidance calls for 20% annual coverage.  If the region/state falls into an exception based on regional commitments, the region should fill in reserved items under 1r to indicate actual commitments or agreements.","20% annual inspection coverage for LQGs.","Active LQGs","State-only (but separate combined % also provided)","RCRAInfo","Select all LQGs in RCRAInfo that have been inspected during last full FY. Count each site once regardless of inspection frequency within the timeframe.","IDEA","NPG","Goal","The denominator makes use of the RCRAInfo Version 3 activity status flag. The March 21, 2005 Site ID Guidance document (RCRA Subtitle C EPA Identification Number, Site Status, and Site Tracking Guidance), posted in RCRAInfo, contains detailed activity status information. Regions and states should have their RCRA active universe defined in RCRAInfo. Trade agreements that are negotiated under national guidance should be tracked using 1r below. Note re: universe counts: OECA researching option of using end-of-year universe where universe counts are required.",null,"b"]
["c","Five-year inspection coverage - Active Large Quantity Generators","National guidance calls for 100% inspection coverage of LQGs over 5 years.","All LQGs should be inspected within 5 years, though if below the 100%, some may be attributed to universe changes or other reasons.","Active LQGs","State-only (but separate combined % also provided)","RCRAInfo","Select all LQGs in RCRAInfo that have been inspected during past five FYs. Count each site once regardless of inspection frequency within the timeframe.","IDEA","NPG","Goal","The denominator makes use of the RCRAInfo Version 3 activity status flag. The March 21, 2005 Site ID Guidance document (RCRA Subtitle C EPA Identification Number, Site Status, and Site Tracking Guidance), posted in RCRAInfo, contains detailed activity status information. Regions and states should have their RCRA active universe defined in RCRAInfo. Trade agreements that are negotiated under national guidance should be tracked using 1r below. Note re: universe counts: OECA researching option of using end-of-year universe where universe counts are required.",null,"c"]
["d","Five-year inspection coverage - Active Small Quantity Generators ","This is an informational metric showing the percent coverage. ","None","Active SQGs","State-only (but separate combined % also provided)","RCRAInfo","Select all SQGs in RCRAInfo that have been inspected during past five FYs. Count each site once regardless of inspection frequency within the timeframe.","IDEA",null,"Informational-only","Inspection presence at smaller sites is important, and this metric provides information to regional reviewers. The denominator makes use of the RCRAInfo Version 3 activity status flag. The March 21, 2005 Site ID Guidance document (RCRA Subtitle C EPA Identification Number, Site Status, and Site Tracking Guidance), posted in RCRAInfo, contains detailed activity status information. Regions and states should have their RCRA active universe defined in RCRAInfo. Note re: universe counts: OECA researching option of using end-of-year universe where universe counts are required. ",null,"d"]
["e","Inspections performed at sites other than those listed in 1a-d in last five years","This is an informational metric showing the number of other inspections performed. ","None","All sites other than those listed in 1a-d (using active universe)","State-only (but separate combined # also provided)","RCRAInfo","Select all sites other than those listed in 1a-d that have been inspected during the past five FYs. Count each site once regardless of inspection frequency within the timeframe.","IDEA",null,"Informational-only","This metric helps provide a complete picture of state evaluation activity.",null,"e"]
["r","Percent of planned inspections completed","Regions can track yearly commitments or multi-year plans.  Reserved for inspection plan targets negotiated between the region and state.",{"$nulls":8},"If states have better active universe numbers than EPA provides in 1a-1d, state-derived results can be put in 1r; however, the region should ensure that the states update and maintain the RCRAInfo active universe.  Also, trade agreements that are negotiated under national guidance should be tracked here.",null,"r"]
["2. Degree to which inspection reports and compliance reviews document inspection findings, including accurate description of what was observed to sufficiently identify violation.",{"$nulls":12},2]
["a","Percentage of inspection reports that are adequately documented in files.","Evaluated based on file review protocol.",null,null,"State-only",{"$nulls":4},"Goal"]
["3. Degree to which inspection reports are completed in a timely manner, including timely identification of violations.",{"$nulls":12},3]
["a","Percentage of Compliance Monitoring Reports/Inspection Reports which identify violations in the file within a given time frame established by the Region and state.","Evaluated based on file review protocol.  150 days to ID SNCs.",null,null,"State-only",{"$nulls":4},"Review Indicator"]
["4.  Degree to which significant violations (e.g., significant noncompliance) and supporting information are accurately identified and reported to EPA national databases in a timely and accurate manner.",{"$nulls":12},4]
["a","SNC identification rate","This measure assesses whether the region/state are actively identifying SNC problems from evaluations.  Numerator=new SNCs in last FY, Denominator=evaluated sites in last FY.","Further discussion recommended for regions/states below 1/2 of the national average.","All sites","State-only (but separate combined % also provided)","RCRAInfo","Select all sites identifed as new SNCs in last full FY.   Compare as a percentage to the number of sites inspected during last full FY.","OTIS Management Reports","ERP","Goal","This is a pointer to increased file review. When below half of the national average, it is recommended to review previous enforcement actions and cases in the pipeline to determine whether SNC did occur but was not reported. ",null,"a"]
["b","Timely SNC determinations","% of SNC determinations (SNY date) completed within 150 days of \"Day Zero\" (first day of inspection) in last FY.","Goal is 100%","All SNCs","State-only","RCRAInfo ","Will be programmed after a year of data is in RCRAInfo Version 3 (FY07).","IDEA","ERP and Data Appendix","Goal","2003 RCRA ERP calls for all SNC deteminations to be made within 150 days of day zero.  Data flow in RCRAInfo Version 3.  EPA will develop analytic approach to handle SVs converted to SNCs, and discuss the possible need for RCRAInfo to require secondary violator designation.",null,"b"]
["c","No activity indicator - new SNCs","No SNCs identified by the state in the fiscal year.","If SNC determinations are not being made, the region should evaluate whether the ERP is being followed.","All sites","State-only","RCRAInfo","Select all sites identified by the state as new SNCs in last FY.  ","OTIS Management Reports","ERP","Review Indicator","Region should examine state enforcement cases to determine whether SNCs were not identified.",null,"c"]
["d","SNC reporting indicator (actions receiving SNC listing)","Metric computes the % of formal actions taken during the FY that received a prior SNC listing, and benchmarks it to national average.","The RCRA ERP discusses the criteria for listing SNC. SNC normally requires a formal action. Some formal actions are taken at non-SNC, so there is not an expectation that 100% of sites with actions had SNC. File review recommended for regions/states below 1/2 of the national average.","All sites with actions","State-only","RCRAInfo","This measure gives the percentage of state formal enforcement actions taken during the fiscal year at facilities that were in SNC in the fiscal year or in the six months prior. (After this info populated in RCRAInfo V.3, this can look for all enforcement actions without a populated \"Addressed SNC Dated\" field.)","IDEA","ERP","Review Indicator","Review is recommended for states below half of the national average, though other states appear to have problems in this area, yet are not at half below the national average. There may be legitimate circumstances for sites to make this list; for example, if states take formal actions against all SVs. Metric will assist region when selecting files for review. Corrective action excluded.",null,"d"]
["5. The degree to which state enforcement actions include required corrective or complying actions (injunctive relief) that will return facilities to compliance in a specific time frame.",{"$nulls":12},5]
["a","Percentage of formal state enforcement actions that contain a compliance schedule of required actions or activities designed to return the source to compliance.  This can be in the form of injunctive relief or other complying actions.","Evaluated based on file review protocol using EPA equivalent state penalty for assessing injunctive relief.",null,null,"State-only",{"$nulls":4},"Review Indicator"]
["b","Percentage of actions or responses other than formal enforcement that return source to compliance.","Evaluated based on file review.",null,null,"State-only",{"$nulls":4},"Review Indicator"]
["6. Degree to which a state takes timely and appropriate enforcement actions, in accordance with policy relating to specific media.",{"$nulls":12},6]
["a","Timely action taken to address SNC","% of enforcement action/referral to DOJ/AG that have been taken within 360 days of Day Zero. Measured as number of SNC sites NOT exceeding 360 days from day zero over number of total SNCs in state. Note that ERP policy allows 20% of SNCs to exceed 360 timeliness milestone.","Of all enforcement actions taken within the fiscal year at SNC sites, 80% of SNC sites should receive a formal action or referral within 360 days of Day Zero. ","SNC sites","State-only (but separate combined % also provided)","RCRAInfo","Will be programmed after a year of data is in RCRAInfo Version 3.","IDEA","ERP","Review Indicator","Data provided in reviews after RCRAInfo Version 3 in production for one year (FY07). The ERP provides 150 days for informal action, 240 days for an initial formal action, and 360 days for final formal action or referral to the State AG. ERP is effective for all civil activities initiated after it went into effect in 2003. Corrective action excluded.",null,"a"]
["b","No activity indicator -  formal actions","No formal actions taken by state in fiscal year.",null,"All sites","State-only","RCRAInfo","Select all state formal enforcement actions in last FY.  ","OTIS Management Reports","ERP","Review Indicator","If this indicates no actions, then look to see if state takes other sanctions for these violations. Corrective action excluded.",null,"b"]
["7. Degree to which a state includes both gravity and economic benefit calculations for all penalties, using the BEN model or similar state model (where in use and consistent with national policy).",{"$nulls":12},7]
["a","Percentage of formal enforcement actions that include calculation for gravity and economic benefit consistent with applicable policies.","File Review based on national program policy and BEN model or state models in use and consistent with national policy.",null,null,"State-only",{"$nulls":4},"Goal"]
["8. The degree to which penalties in final enforcement actions include economic benefit and gravity in accordance with applicable penalty policies.",{"$nulls":12},8]
["a","No activity indicator - penalties","No penalties taken by state in fiscal year.",null,"All sites","State-only","RCRAInfo","Select and sum the dollar amount of all penalties (final assessed penalties and SEP credits) at state enforcement actions (in 200,239 or in 250,339 or in 350,539 or in 550,799) in last FY.  ","OTIS Management Reports","Required data field in RCRAInfo","Review Indicator","If this indicates no penalties, then look to see if state takes other sanctions for these violations. Corrective action excluded.",null,"a"]
["b","Penalties normally included with formal enforcement actions","8b1. % of formal enforcement actions that carry any penalty in last FY.","Program review necessary if state is below one half of the national average for % of actions with penalty.","All sites with state formal actions","State-only","RCRAInfo","Select all state formal enforcement actions (in 200,239 or in 250,339 or in 350,539 or in 550,799) with penalties (final assessed penalties and SEP credits) in last FY.  Compare to the total number of state formal enforcement actions in last  FY.","IDEA",null,"Review Indicator","This metric does not measure appropriateness of penalties, but does flag when additional review is necessary.  If no penalties, then look to see if state takes other sanctions for these violations. Note that under the RCRA ERP not all non-SNC-related actions require a penalty. The action may include other sanctions depending on the situtation and may not include a monetary penalty. Corrective action excluded.",null,"b"]
[null,null,"8b2. % of final formal enforcement actions that carry any penalty in last FY.","Program review necessary if state is below one half of the national average for % of actions with penalty.","All sites with state final formal actions","State-only","RCRAInfo","Select all state final formal enforcement actions (in 300,339 or in 350,399 or in 500,539 or in 550,699) with penalties (final assessed penalties and SEP credits) in last FY.  Compare to the total number of state final formal enforcement actions in last FY.","IDEA",null,"Review Indicator"]
["9. Degree to which enforcement commitments in the PPA/PPG/ categorical grants (written agreements to deliver a product/project at a specified time) are met and any products or projects are completed.",{"$nulls":12},9]
["a","State agreements (PPA/PPG/SEA, etc.) contain enforcement and compliance commitments that are met.","Review of PPAs, PPGs, SEAs, etc.",null,null,"State-only",{"$nulls":4},"Review Indicator"]
["10. Degree to which the Minimum Data Requirements are timely.",{"$nulls":12},10]
["a","Integrity of SNC data (timely entry)","Percent of SNCs that are entered to RCRAInfo more than 60 days after the determination.  Measures the \"lag\" between the date of SNC determination and the actual reporting of the SNC determination to RCRAInfo.","ERP states that the data should be entered when the determination is made. SNC entry should not be withheld until the action is completed. EPA expects SNC data to be entered more quickly than 60 days, so this metric provides some \"cushion\".","All SNCs","State-only","IDEA using RCRAInfo data","When a new SNC is seen by IDEA, it is known that the data was entered since the previous monthly RCRAInfo extract (does not provide the exact date). For analytic purposes, it is assumed that newly reported SNCs were put into RCRAInfo on the date of the previous extract. This date is compared to the date entered into RCRAInfo as the SNC (SNY) determination date. ","IDEA","ERP and Data Appendix","Review Indicator","If a problem exists in this area, it negatively impacts timeliness tracking under the ERP, the Watch List, award program screening, and public access. There may be legitimate circumstances for sites to make this list, but the region should examine this issue to determine these reasons are valid. Minor additional time lags may occur for translator states.",null,"a"]
["b","Percent of inspections, enforcement actions, or other compliance or enforcement related activities for which there is a Nationally Required Data Element, that are entered into RCRA INFO in a timely manner.","File review","Regions should evaluate what is maintained in RCRAInfo by the State and ensure that all nationally required data elements are properly tracked and entered according to accepted schedules.",null,"State-only",{"$nulls":7},"b"]
["11. Degree to which Minimum Data Requirements are accurate.",{"$nulls":12},11]
["a","Integrity of SNC data (correct entry of SNC determination date)","11a1. Measures \"closeness\" between SNC determination and formal enforcement action.  Shows the number of sites that were \"SNC determined\" on same day as formal action.","ERP states that the data should be entered when the determination is made SNC entry should not be withheld until the action is completed.","All SNCs that have reached formal action","State-only","IDEA using RCRAInfo data","This measure compares the date of the SNC with the date of the first formal action that came after the SNC.  This method does make some assumptions (e.g., that the action taken doesn’t address a previous SNC). ","IDEA","ERP and Data Appendix","Review Indicator","If a problem exists in this area, it negatively impacts timeliness tracking under the ERP, the Watch List, award program screening, and public access. There may be legitimate circumstances for sites to make this list, but the region should examine this issue to determine these reasons are valid. ",null,"a"]
[null,null,"11a2. Measures \"closeness\" between SNC determination in last FY and formal enforcement action.  Shows the number of sites with SNC determination within one week of formal action.","ERP states that the data should be entered when the determination is made SNC entry should not be withheld until the action is completed.","All SNCs that have reached formal action","State-only","IDEA using RCRAInfo data","This measure compares the date of the SNC with the date of the first formal action that came after the SNC.  This method does make some assumptions (e.g., that the action taken doesn’t address a previous SNC). ","IDEA","ERP and Data Appendix","Review Indicator","If a problem exists in this area, it negatively impacts timeliness tracking under the ERP, the Watch List, award program screening, and public access. There may be legitimate circumstances for sites to make this list, but the region should examine this issue to determine these reasons are valid. "]
["b","Longstanding secondary violations not \"returned to compliance\" or redesignated as SNC.","Number of sites in violation for greater than 3 years.  This measure is designed to ensure that violations are given an end date.","The ERP states that sites designated as secondary violators should be re-designated as SNC if the violator does not return to compliance in 240 days.","All sites","Combined","IDEA using RCRAInfo data","Select RCRA sites in violation greater than three years, but not currently in significant noncompliance.\n","OTIS RCRA interface (select facilities in violation greater than 3 years and not SNC)","Mandatory Data Elements in RCRAInfo","Data Quality","The region should review the site list and determine whether data quality issues exist that need to be addressed or whether SNC designations need to be made. This uses 3 years because that matches the OTIS data pull; however, guidance has a tighter 240-day standard.",null,"b"]
["12. Degree to which the minimum data requirements are complete, unless otherwise negotiated by the region and state or prescribed by a national initiative.",{"$nulls":12},12]
[null,"For the following elements, the region should provide the counts to the state from the OTIS State Review Framework data metrics site and ensure the state agrees with these counts. If there is disagreement, additional evaluation should be conducted to determine the source of the discrepancy and develop an action plan for making appropriate corrections."]
["a","Active site universe counts accurate","12a1. Number of operating TSDFs in RCRAInfo ","Region and state should agree on operating universe count.",null,"State-only","RCRAInfo","Select all operating TSDFs in RCRAInfo. Uses the current RCRAInfo definition of operating TSDF.","IDEA","Required data field in RCRAInfo","Data Quality",null,null,"a"]
[null,null,"12a2. Number of active LQGs in RCRAInfo","Region and state should agree on active universe count.",null,"State-only","RCRAInfo","Select all active LQGs in RCRAInfo.","IDEA","Required data field in RCRAInfo","Data Quality","12a2-12a4 use the RCRA Version 3 activity status flag."]
[null,null,"12a3. Number of active SQGs in RCRAInfo","Region and state should agree on active universe count.",null,"State-only","RCRAInfo","Select all active SQGs in RCRAInfo.","IDEA","Required data field in RCRAInfo","Data Quality"]
[null,null,"12a4. All other active sites in RCRAInfo","Region and state should agree on active universe count.",null,"State-only","RCRAInfo","Select active sites in RCRAInfo other than those listed in 12a1-12a3.","IDEA",null,"Data Quality"]
["b","Inspection counts complete","12b1. Number of inspections performed by state during reporting period. ","Mandatory data elements are listed in the RCRAInfo documentation.","all","State-only","RCRAInfo","Select all inspections (HQCAC,HQCDI,HQCEI,HQGME,HQFRR, HQNRR, HQCSE, HQOAM, HQFCI) performed by state during FY.","OTIS Management Reports","Required data field in RCRAInfo","Data Quality",null,null,"b"]
[null,null,"12b2. Total # of sites inspected during reporting period","Mandatory data elements are listed in the RCRAInfo documentation.","all","State-only","RCRAInfo ","Select all sites inspected (HQCAC,HQCDI,HQCEI,HQGME,HQFRR, HQNRR, HQCSE, HQOAM, HQFCI) by state during FY.","IDEA","Required data field in RCRAInfo","Data Quality"]
["c","Violation counts complete","Number of sites with violations during reporting period","Mandatory data elements are listed in the RCRAInfo documentation.","all","State-only","RCRAInfo","Select all facilities with secondary violations in FY.","OTIS Management Reports","ERP and RCRAInfo","Data Quality","This metric should count violations for out-of-state sites that are resolved by a state. ",null,"c"]
["d","Notice of violation counts complete","12d1. sites with state NOVs (informal enforcement actions)","Mandatory data elements are listed in the RCRAInfo documentation.","all","State-only","RCRAInfo","Select sites with actions in 100,199 during FY.","OTIS RCRA query","ERP and RCRAInfo","Data Quality",null,null,"d"]
[null,null,"12d2. total state NOVs (informal enforcement actions) issued","Mandatory data elements are listed in the RCRAInfo documentation.","all","State-only","RCRAInfo","Select actions in 100, 199 during FY.","OTIS Management Reports","ERP and RCRAInfo","Data Quality"]
["e","SNC counts complete","12e1. # of new SNCs in last FY; ","Mandatory data elements are listed in the RCRAInfo documentation.","all","State-only","RCRAInfo ","Select sites with new state SNYs in FY.","OTIS Management Reports","ERP and RCRAInfo","Data Quality",null,null,"e"]
[null,null,"12e2. # of sites in SNC in last FY","Mandatory data elements are listed in the RCRAInfo documentation.","all","State-only","RCRAInfo","Select sites with an SNY in FY,  or an SNY in a previous FY and no SNN before the start of the FY.","IDEA","ERP and RCRAInfo","Data Quality","Some states have different practices regarding turning off the SNC designation in RCRAInfo."]
["f","Formal action counts complete","12f1. sites with formal actions.","Mandatory data elements are listed in the RCRAInfo documentation.","all","State-only","RCRAInfo","Select sites with formal actions in 200,239 or in 250,339 or in 350,539 or in 550,799.","OTIS Management Reports","ERP and RCRAInfo","Data Quality","Corrective action excluded.",null,"f"]
[null,null,"12f2. total formal actions taken.","Mandatory data elements are listed in the RCRAInfo documentation.","all","State-only","RCRAInfo","Select formal actions in 200,239 or in 250,339 or in 350,539 or in 550,799.","OTIS Management Reports","ERP and RCRAInfo","Data Quality","Corrective action excluded."]
["g","Assessed penalties complete","Total amount of final (assessed) penalties","Mandatory data elements are listed in the RCRAInfo documentation.","all","State-only","RCRAInfo","Select and sum the dollar amount of all penalties  (final assessed penalties and SEP credits) at state enforcement actions (in 200,239 or in 250,339 or in 350,539 or in 550,799) in last FY.  ","OTIS Management Reports","ERP and RCRAInfo","Data Quality",null,null,"g"]