The 2014 NERC audit season is soon upon us. While some Registered Entities used Q4 2013 to get a jumpstart on audit preparations, others will return from the year-end holidays either never having been through an audit or having years pass since their last sit-down with Regional auditors. To help ensure a successful NERC audit, this article focuses on 3 critical areas within Reliability Standard Audit Worksheets – preparation, narrative development, and evidence documentation. Our examination of these areas is supported by RSAW examples, best practices, and the experience of a Generation Operator whose proactive RSAW preparation was key to ensuring a successful audit.
RSAWs are the Road Map of Your Compliance Program
As Reliability Standard Audit Worksheets (RSAWs) are used by NERC Compliance Enforcement Authorities (CEAs) when auditing Registered Entities, the RSAW is your primary method of communicating your entity’s internal compliance process, controls, and evidence. As such, RSAWs are the ‘make it or break it’ component of your audit – they are the road map of your compliance program as:
- they determine whether or not sufficient evidence supports your compliance with applicable NERC standards and requirements
- they are the tool for identifying non-compliances and potential violations
- they serve as a public record of your audit, and they help you prepare for an audit and, if maintained up-to-date (“Living” RSAWs), provide for on-going compliance maintenance.
5 Common RSAW Challenges
Over the past few years, Regional and Registered Entities have been sharing NERC audit experiences and best practices. While helpful, they are often broad discussions of “Here’s what was required, here’s what we did, make sure you address that” type of presentations.
However, when Notification of Audit letters arrive in your inbox defining the audit scope and sub-requirements, those PowerPoint discussions simply don’t go deep enough into the real challenges of responding to RSAWs. While our readers can undoubtedly add to our list, some common RSAW challenges you may face include:
| RSAW Knowledge Gaps Between Audits | Completing RSAWs accurately can be challenging due to changing NERC requirements, critical knowledge lost through retirement and job changes, and escalating resource constraints. With the exception of spot checks, regulatory audits of Registered Entities (RE) are typically 3 or more years apart. As a result, there may not be an on-going process of maintaining RSAWs up– to-date – thus preventing dynamic or 'living' compliance. |
| The Difficulty of RSAW Preparation | RSAWs demand adherence to exacting policy, procedure, narrative, evidence, formatting, and submittal package requirements. They are not easy – or quick – and require exacting attention to detail. Ensuring successful RSAW preparation and submittal has required hundreds of hours for some Registered Entities. |
| Inconsistent Standard Applied to RSAW Narratives During Regional Audits | Consistent measure applied to RSAW narratives, evidence type, or submittal across the eight Regional Entities is often challenging. Registered Entities in multiple jurisdictions must ferret out Regional differences and requirements. Auditors often rely more upon evidence provided and discussions or questioning to determine compliance instead of the RSAW narrative. Expectations for RSAW narratives vary by the audit team. (NERC and the Regional Entities recognize this and are working to address it). |
| No Central Repository for Files, Folders, & Evidence | Often, compliance documentation is archived in various Word documents, stand alone Excel files, personal emails, PDFs, phone texts, voice mails, and legacy enterprise systems – few of which are searchable, available anywhere/ anytime, or electronically linked. Often, there are multiple versions of policies and procedures, narratives, and evidence, and RSAWs may be out of date, differing in format, and/or content. Without a central compliance management system, Registered Entities must often 'hunt down' and verify hundreds of documents and information artifacts (there is rarely a 'map' of where things are scattered) and create a standard management system. |
| Inconsistent RSAW Submittal Package Requirements | RSAWs often must be submitted via specific versions of Internet Explorer, Mozilla Firefox, or Chrome. Interfaces to EFT Server / Client interfaces may be required, along with specific applications like Java Runtime Environment V6. File structure and naming conventions can be confusing and vary by Regional Entity. (In future articles, we will address RSAW submittal package requirements). |
Getting Down to the Real Nitty-Gritty: RSAW Preparation, Narrative, and Evidence Documentation
Audit schedules are published by the Regional Entities each year, with Initial Notification Letters and Compliance Surveys typically sent six months in advance of the audit date. Detailed audit letters are often sent three months in advance of the audit.
Well-prepared RSAWs demonstrate a company’s commitment to a thriving ‘compliance culture’ in advance of the actual audit date. Being prepared 30 to 60 days ahead of these milestones is recommended. Here are some areas to think through (preparing to prepare) before making assignments, gathering files, and writing responses:
Step 1: RSAW Preparation – Things to Consider Upon Audit Notification
When should we start preparing?
- When is the audit?
- What are our expectations for RSAW preparation and submittal?
- When should we kick off the RSAW process?
Where should we start?
- Do we have multiple, decentralized business areas that are involved in the audit?
- Do our facilities cross Regional Entity jurisdictions?
- Have we delegated any reliability standard requirements to another entity?
- Have any reliability standard requirements been delegated to us from another entity?
- Have the standards changed?
- Have the RSAW requirements changed – Do we have the most up-to-date versions?
- Have the Regional Entities’ expectations for RSAW completion changed?
- What is our communication plan with the regulator?
Who needs to be involved?
- Who has previous RSAW and audit experience?
- Who understands Regional Entity differences?
- Who are specific Subject Matter Experts?
- Who should be on the RSAW prep team?
- Who needs training and indoctrination on the audit and RSAW development process?
How will we manage the documents?
- What policy and procedure documents, spreadsheets, emails, messages, etc, are needed for this RSAW?
- What are the RSAW formatting requirements?
- How and where are we managing the process electronically?
Step 2: RSAW Narrative Development
Now that you’ve thought through your RSAW approach and located the appropriate documents, you are ready to develop clear and concise RSAW narratives – narratives that support compliance, while not providing unrelated or superfluous information. Here are four principles to keep in mind as you do:
- Clarity: Auditors may only look at evidence rather than reading the RSAW narrative, or the Auditor may read the paragraph narrative and may or may not understand your meaning. Your narratives must be very clear and concise.
- Consistency: Be consistent when a document is referenced repeatedly or throughout multiple RSAW narrative responses in terms of naming conventions and file descriptions.
- Accuracy: Is the narrative accurate? Did content get overlooked or embellished?
- Full Disclosure: All narratives and documentation must be complete – regardless if they are favorable or not. Remember to include compliance by others to whom you have delegated compliance.
Next, RSAW narrative requirements have been changing – requiring Registered Entities to more clearly depict the linkages between the standards, requirements, and narratives to specific evidence in the requirement.
To illustrate these points, below are two examples of RSAW narratives across two years - as narrative requirements change - with best practices noted with letters:
Best Practices
- Narrative responses should follow the sequence of the requirements and sub-requirements – not restate them.
- Procedures that support RSAWs should contain sufficient detail to tie to the NERC standard or requirement.
- Elaborate in concise detail how you comply with the requirement. Provide descriptions of policies and procedures with reference to evidence.
- Well organized and clearly presented information facilitates the Auditor’s verification process and minimizes Requests for Additional Information.
Best Practices
- Elaborate in concise detail how you comply with the requirement. Provide descriptions of policies and procedures with reference to evidence.
- Procedures that support RSAWS should contain sufficient detail to tie to the NERC standard or requirement.
- Elaborate in concise detail how you comply with the requirement. Provide descriptions of policies and procedures with reference to evidence.
- Well organized and clearly presented information facilitates the Auditor’s verification process and minimizes Requests for Additional Information.
Operator’s Proactive Approach to RSAW Management ensured no NERC Audit ‘Surprises’
Nelson Industrial Steam Company (NISCO) provides 260 MWh of electricity to areas of the Southeast U.S. In April of 2011, the Southeast Electric Reliability Council (SERC) notified NISCO of their first NERC audit. SERC’s detailed Audit Compliance Letter listed eight standards that would be audited, along with review of 58 sub-requirements, evidence of an Internal Compliance Program, and a current organization chart.
Prior to SERC’s notification, NISCO recognized the importance of proactively organizing Reliability Standard Audit Worksheets (RSAW) policies, procedures, and evidence documentation well in advance of the audit period.
“As NISCO’s Compliance Manager,” says Mr. Shelley Hacker, NISCO’s Site Operations Manager, “I have found that accurate, complete, and concise RSAWs are essential to my success during an audit cycle. At audit time, we want no surprises.
“Audit success should be based upon the sum of many small efforts, repeated day in and day out, and not just prior to an audit’s conduct,” continued Hacker. “Our audit preparation required hundreds of hours, but we were not overwhelmed because of our early engagement and a defined internal compliance program. This experience highlighted the following factors as being essential for ensuring the quality of our RSAW responses, the supporting audit package, and ultimately successful audit:”
- Identify the applicable reliability standards and requirements applicable to registration as a Generator Owner.
- Assign Subject Matter Experts for each NERC Reliability Standard and Requirement.
- Define the document file structure and file naming before beginning the RSAW response process.
- Update required policies and procedures and maintain up-to-date versions.
- Develop accurate and concise RSAW compliance narratives that support compliance with each requirement and sub requirement, and that include the rationale for the conclusions reached, yet do not provide unrelated or superfluous information.
- Link appropriate evidence to each RSAW requirement or question’s narrative and summary of evidence tables.
- Assemble all RSAWs and audit submittal information in the manner prescribed by the regional entity – as it is not always consistent between audit teams or regional entities.
- Ensure that appropriate resources are applied throughout the organization to support day-to-day compliance activities, as well as to support audits.
- Use a compliance-specific web-based document management as a central repository to streamline the RSAW process, link all electronic documents and automate the RSAW Package Submittal process.
- Maintain up to date RSAWs between audit activities for ongoing compliance management.
Audit Results every Registered Entity Strives For
The SERC audit team thoroughly reviewed documentation provided by NISCO. Data, information, and evidence submitted in the form of policies, procedures, emails, logs, studies, data sheets, etc., were validated, substantiated, and cross-checked for accuracy as appropriate. As a result of NISCO’s early and proactive RSAW preparation, SERC’s onsite audit of NISCO was highly successful.
In both examples, notice that the author avoided immaterial content that, while related, is not applicable or out of scope. This may include referencing data, policies, or procedures that don’t support compliance to the Standards or sub-requirements and avoiding assumptions and ‘interpretations’ of the standards – unless they are material and can be validated by evidence. The author also made sure that all documents were the final and approved versions and that un-approved content was not overlooked.
Step 3: RSAW Evidence Best Practices
Every RSAW requires submission of an evidence matrix that validates your compliance narrative. Your evidence needs to be carefully organized, accurate, and citations linked to specific requirements and sub-requirements within your narrative.
To illustrate this, below is an example of evidence best practices for the 2012 VAR- 002.1.1b / Generator Operation for Maintaining Network Voltage Schedules RSAW shown above:
Best Practices
- The date/revision should be indicated as applicable.
- Evidence files should be developed so as to clearly indicate what the evidence file is by just reading its title as it relates to the RSAW narrative.
- The files in this table are linked directly to the files so that the evidence can be viewed by simply ‘clicking’ on the file name.
RSAW Next Steps
Once the RSAW narrative and evidence components are complete, conduct a detailed final review with all Subject Matter Experts:
- Was the document spell-checked?
- Are all narratives and evidence linked?
- Is anything missing?
- If completing the RSAW for multiple sites, were there any changes to versions of the RSAW that were missed?
- Did you have someone with strong editing skills check the document for the required formatting?
- Did you archive all compliance documentation in a central repository so that each RSAW can become a ‘living’ document easily kept up to date?
Submittal Package Requirements
Once your RSAW compliance narratives and evidence validation are complete, it’s time to ready the RSAW package for electronic submittal. In future articles, we will discuss RSAW Submittal Package requirements – as they are worthy of their own examination.
About the Author
Bob Biggs has more than 35 years of utility experience in generation plant operation and maintenance (fossil, hydro, nuclear, and wind), protective systems, selfassessment programs, facility ratings, and regulatory compliance. He deeply understands the regulatory lifecycle of NERC standards, development, regulatory policies and procedures, Regional Entity audits, findings, enforcement, and mitigation. Formerly the head of Entergy’s Electric Reliability Standards Corporate Compliance Division, Bob is the Services Manager and currently serves as Office of NERC Compliance Manager for Certrec – a leading regulatory compliance expert that helps utilities manage the regulatory process to their advantage through a suite if Internal Compliance Program solutions. For more information please visit www.certrec.com
