• about NWI
  • home
  • mission
  • functions
  • organization
  • history
  • partners/funders
  • about our resources
  • co-directors/staff
  • contact us
• resources
  • wraparound basics
  • resource library
  • webinars
  • resource guide
  • assessment/fidelity
  • implementation support
    • implementation overall
    • community partnership
    • collaborative action
    • finance/sustainability
    • supports/services
    • human resource support
    • accountability
    • state support
  • consultants
• publications
  • theory of change
  • ten principles
  • phases and activities
  • implementation guide
  • user's guide
  • manual de usario
  • family partner/principles
  • family partner/activities
  • family partner/qualifications
  • nwi impact
  • evidence base
  • bibliography
  • wraparound research
  • implementation overview
  • wraparound comix
  • order publications
• news & events
  • wraparound news
  • wraparound events
  • news from the field
• wrap USA
  • nwi across USA
  • wraparound registry
• join NWI!
• members
  • member home
  • member news
  • discussions / groups
  • member directory
  • newsletter
  • member support

accountability*

*For more in-depth information on this aspect of implementation, please refer to the NWI's Implementation Guide.

As conceived by the National Wraparound Initiative, implementation of wraparound requires attention to six types of community supports. One of these areas is Accountability.

According to the Community Supports for Wraparound Inventory, when a wraparound initiative is fully supported in the area of Accountability, the community has implemented mechanisms to monitor wraparound fidelity, service quality, and outcomes, and to assess the quality and development of the overall wraparound effort.

The resources in this section provide information regarding how stakeholders involved in the wraparound effort take concrete steps to implement mechanisms to monitor wraparound fidelity, service quality, and outcomes, and to assess the quality and development of the overall wraparound effort. Questions addressed include:

1. What are the key issues to consider in building accountability for our wraparound project?
2. What are the implementation steps we should undertake to build accountability?
3. What sort of data do we gather to assess whether or not we are doing high quality wraparound?
4. What sort of outcomes do communities typically measure?
5. What types of Management Information Systems (MIS) are needed to support accountability?
6. What are core steps to take to ensure accountability?
7. Where can I find more resources on accountability?

1. What are the key issues to consider in building accountability for our wraparound project?

Communities implement Wraparound for a variety of different reasons. One community may be concerned about spending patterns and a sense of little or no outcome for monetary investment. A second community may be more focused on how the community of service providers meets children’s and families’ needs. Rather than just measuring things that have been measured by other projects, or picking evaluation instruments “off the shelf,” true accountability in wraparound requires asking what the goals of the project are, how we will know we have achieved these goals, and linking measurement strategies to these information needs. To achieve this kind of accountability, it is important to start early on the in the process of implementation, with wraparound stakeholders working together to establish indicators of success and failure. Areas to consider in building accountability include:

• Establish clear outcomes. This area answers the question of whether you are getting the right results for your effort. Stakeholders in the wraparound initiative should have an opportunity to collectively establish what outcomes are most important to them. If you don’t know the desired result then you run the risk of practicing for process’ sake rather than practicing with a purpose to get a desired outcome. You also won’t be able to answer questions about whether your effort has succeeded.

• Set defining process elements. This area answers the question of whether you are following the right implementation procedures in your wraparound project. Similar to establishing outcomes, a challenge for any new project is identifying what key process elements must be reliably achieved for participating youth families. This means that your wraparound project has to first decide what practices you want staff and managers to follow and then monitor to make sure those practices are followed. There is no single best way to measure the process of wraparound implementation. Some sites will use research tools such as the Wraparound Fidelity Assessment System (WFAS) to get information about whether the activities and principles of wraparound are being followed. As an alternative—or in addition—to WFAS or other research tools, sites may use supervisory and program checklists that identify how the process is being implemented at the youth and family level.

• Gather satisfaction and other data directly from youths and families. This area answers the question of whether individual youth and families are satisfied with your implementation of wraparound. Simply following the process or steps of wraparound implementation does not guarantee that youth and families will be satisfied with the process or that they will be getting their needs met. This area of building accountability requires direct feedback from youth and families who are most affected by the project. Such information may focus on their satisfaction with the wraparound process, satisfaction with services that are being received, and perceptions of whether needs are being met, whether progress is being made, and what barriers that are getting in the way. Such information can be gathered through written surveys or direct interviews and should occur regularly from the onset of the project. Some sites will contract with family organizations or train family members to gather this information.

• Monitor costs. This area answers the question of whether your investment of time, money, personnel, space and other resources is worth it. There is no national standard for pricing wraparound with wide regional variances existing throughout the country. Costs associated with wraparound include care coordination costs associated with arranging and organizing the process and costs associated with the necessary services, supports and strategies that are outlined in a wraparound plan of care.

The four dimensions above come together to answer a complex question: Are you realizing the right results for doing the right things because you’ve made the right investments that satisfy the young people and families you are serving and successfully meet their needs? At the same time, there are complex issues that may facilitate or hinder success that are difficult to measure with a simple evaluation tool. For example, the host environment in which your project operates is vitally important to the success of wraparound. Organizational coherence can have a positive impact on staff morale and project costs as well as family outcome. If you are not realizing the types of results you were hoping for within each of these areas, you may find that the issue is lack of coherence with the host environment. This means that wraparound is not likely to thrive in an organization or system that doesn’t align with wraparound principles. In addition to the above issues, it is important to be aware of the degree to which there is a fit between the organization, the system, and the goals and principles of the wraparound project.

2. What are the implementation steps we should undertake to build accountability?

Before making decisions on specific instruments or measurement strategies, building accountability for a wraparound initiative requires adherence to several core considerations.

• Determine how you will use your information before you begin to collect it. The point of data collection is not just to collect data but to actually use it. It often helps to come up with a simple answer to the key question of “what information do we need and how will we use it or gain from it?” Once that is established you can then identify what information you need that will get you where you need to go, and what the best source(s) of data will be.

• Be critical in your methods. Some projects will collect information because it’s convenient rather than collecting data that can stand up to scrutiny. Examples of this include surveying only those individuals who are easily reached, forming conclusions based on low response rates or using methods to collect information that would lead one to question validity. An example of that might be relying solely on the provider of a service with the responsibility for data collection. Some could argue this approach will result in a biased response.

• Set reasonable goals for data collection. Wraparound projects should be disciplined in terms of establishing what they want to measure and then ensuring that adequate resources are dedicated to reliable and valid data collection, aggregation, and reporting. Projects should be disciplined about what they need to know and avoid drifting to things they might like to know, especially if it is not a clear need as identified by project stakeholders.

• Stay on the collaborative course. In building a set of data measures it is important to continue to include other stakeholders and perspectives, including youth and families. Different people can get focused on single indicators that they take as a placeholder for quality or success. Reaching consensus about definitions of quality and success – and revisiting these definitions regularly – is one strategy for assuring that key wraparound sponsors have a shared vision of quality and success. It can also help these stakeholders stay together throughout implementation.

• Focus on the meaning of the process not just the steps of the process: As more is written and studied about wraparound, more detail emerges. This detail can lead to a reductionist course in which lots of discrete steps are evaluated but the overall purpose or goal of wraparound is lost. A focus on individual steps poses its own set of problems. Even though you can technically deliver each of the steps, this does not necessarily mean that you are providing quality wraparound. Like ballroom dancers, wraparound staff persons need to be able to put the steps together in a fluid pattern that communicates care, concern and compassion for and on behalf of a family. Additionally, the more steps you measure adds complexity, and means having to focus on deciding what step(s) is/are most important. Though measuring process is important, remember to not lose focus on the meaning of the wraparound process and the overall goals of your project.

3. What types of data do we gather to assess whether or not we are doing high quality wraparound?

As described above, measuring the process of wraparound implementation can take many forms. Data collection and feedback can be critical in the process of supervising and coaching staff, as described in a chapter in the Resource Guide to Wraparound on Wraparound Supervision and Management. Others may focus more on using wraparound fidelity evaluation tools, which can be used in supervising wraparound staff, but are more frequently used in aggregate form to provide feedback to the site and its stakeholders about how implementation is going overall. Wraparound projects need to make these decisions for themselves. In general, however, measures of the process of wraparound tend to provide:

• Data on the quality of the wraparound process provided, collected by live observation, plan review, and feedback from youth and families. The methods used to assess the process and its quality should be grounded in the principles of wraparound and used as the basis for ongoing quality assurance/improvement.
• Monitoring and analysis of the types of services and supports included in wraparound plans, whether or not planned services and supports are provided, and whether or not the goals and needs that appear on wraparound plans are met.
• An ongoing process to track satisfaction and buy-in among stakeholder groups, including youth and families and representatives of partner agencies and organizations.
• An ongoing, systematic process for identifying and addressing barriers that prevent wraparound teams from doing their work and/or fully implementing their plans.

4. What types of outcomes do communities typically measure?

As described above, wraparound projects may define success in many ways, so it is critical to convene a collaborative process to define what represents success in terms of ultimate outcomes. Information on outcomes should be relevant to funding decisions, policy discussions, and strategic planning. Outcomes that are chosen should be important to stakeholders as well as to families and reflect the values of wraparound; for example:

• Enabling children and youth to be “at home, in school, and out of trouble”
• Increasing child and family assets and strengths and reduction of needs
• Improving caregiver well-being
• Increasing family and youth empowerment

Short-, intermediate-, and long-term outcomes that are theoretically important in wraparound are outlined in the Resource Guide article on the theory of change for wraparound. Outcomes that have been used in previous research are described in the article that reviews outcome studies and the article that examines the evidence base for wraparound.

5. What types of Management Information Systems (MIS) are needed to support accountability?

Similar to deciding outcomes and process measures, deciding how to manage information collected in support of a wraparound project will necessarily be driven by local needs and resources. However, well-established wraparound initiatives tend to have MIS systems that can maintain information that serve a range of functions, including maintaining information on youth and families who are enrolled, authorizing services and making payments, and tracking youth and family outcomes. The Resource Guide to Wraparound includes a detailed description of how Wraparound Milwaukee has built and integrated its MIS systems over time.

6. What are core steps to take to ensure accountability?

Accountability is more than just a research design of your wraparound effort. Some sites have relatively large resource pools to access in building and executing a research and data collection design. Other sites have minimal resources and are struggling just to get things aligned for implementation (rather than focusing on evaluation components). Whatever category your project fits, the following steps can be helpful in assuring adequate accountability:

• Define what you want to know:
• Do you want to assure a minimum level of practice?
• Do you want to assure the right fiscal expenditures?
• What outcomes do you hope to achieve?
• How will family perspectives on satisfaction and quality of the process align with the other three questions?
• What decisions will this information inform?
• Define your data source(s):
• Define an existing data source that would meet your information need
• Outline question(s) for which you need additional information
• Fill in the blanks:
• Define what missing information you would need to answer your unanswered question
• Do you have access to the information or will you have to create it?
• Do you have the staff to do that? If not, who will?
• Do you have the resources to do that? If not, where will you get them?
• Do you have the political will to do that?
• Define what your unit of analysis must be?
• Individual families
• Teams
• Staff
• Other
• Establish a process for review:
• Define how frequently your results and information should be reviewed
• Define who should be involved in reviewing them
• Establish your protocol for decision making:
• How will you interpret the data?
• How will you use it to develop shared meaning among your stakeholders?
• Use the data to inform action:
• Stop doing something
• Start doing something else
• Get additional resources
• Get more information

7. Where can I find more resources on accountability?

For more resources, see our collection of additional resources on accountability.

The information on this page has been peer reviewed through the NWI.

2013, National Wraparound Initiative, Portland State University. | For proper viewing of this website, please ensure you are using the latest version of your browser.