Methods for determining the effectiveness of the implementation of additional educational (general educational) programs. On approval of performance indicators of state (municipal) institutions of additional education for children, their heads

Target indicators of efficiency and effectiveness of the activities of the head of the institution

Criteria for evaluating the effectiveness and efficiency of the activities of the head of the institution (unit of measurement - points)

Reporting form containing information on the achievement of the indicator

Frequency of reporting

(100 points)

Compliance with labor discipline and proper performance of labor duties

Execution/non-execution

Quarterly

Availability of written substantiated complaints, appeals containing criticism

Lack of substantiated complaints

Report of the head of the structural unit of the institution

Quarterly

Working with children from socially disadvantaged families

The presence / absence of this category of the population in groups (sections, associations, etc.)

Report of the head of the structural unit of the institution

Quarterly

Implementation of activities that ensure interaction with parents

implemented / not implemented

Report of the head of the structural unit of the institution

Quarterly

Organization (participation) of systematic research, monitoring of individual achievements of students

Running / not running

Report of the head of the structural unit of the institution

Quarterly

Dynamics of individual educational results (based on the results of control measures, intermediate and final diagnostics)

tracked / not tracked

Report of the head of the structural unit of the institution

Quarterly

Participation and results of participation of students in competitions, competitions of the regional, all-Russian and international levels

The share of participants in competitions, competitions of various levels from the total number of students / The share of prize-winners and winners from the total number of participants in competitions and competitions

Report of the head of the structural unit of the institution

Quarterly

Organization of additional projects, in addition to the main activity

implemented / not implemented

Report of the head of the structural unit of the institution

Quarterly

Mastering the programs of advanced training or vocational training

Completion of courses or advanced training programs on time (at least 72 hours)

Report of the head of the structural unit of the institution

Number of developed, published guidelines, collections, booklets, leaflets for the population with the participation of a specialist (trainer); presentation of a specialist with reports at conferences, seminars, meetings, round tables, etc.

Presence/absence of publications

Report of the head of the structural unit of the institution

Quarterly

Ensuring safety during classes and cultural events

No Incidents

Report of the head of the structural unit of the institution

E.N. Nikitin

The article was accepted for publication in June 2008.

annotation

INDICATORS AND TARGET INDICATORS OF THE EFFICIENCY AND PERFORMANCE OF PROVIDING SERVICES IN THE SYSTEM OF GENERAL EDUCATION (on the example of

special education)

The article discusses a general approach to the development of a system of indicators and target indicators for the effectiveness and efficiency of the provision of educational services, taking into account foreign experience. More than 80 indicators collected in 56 special (correctional) educational institutions in seven constituent entities of the Russian Federation were analyzed. Using target indicators and indicators of the provision of educational services to people with disabilities, it is possible to control the effectiveness of budget expenditures in the implementation of the educational policy of the city, district, region. This will allow timely detection of weaknesses in the financing of a standard educational service, control the level of accessibility of quality education, compliance with social justice. The use of a system of indicators and target indicators contributes to the optimization of spending on special education and the efficient allocation of limited budgetary resources.

Reforming the education system in accordance with the new goals and objectives that it faces under the modernization program involves the development and use of a set of indicators and target indicators that allow you to track both the state of the system and ongoing changes, assess the efficiency and effectiveness of the provision of general education services.

E.N. Nikitin

As part of a study conducted by the Center for Universal Programs together with the Center for Social Research and Innovation on the territory of 7 constituent entities of the Russian Federation (Kabardino-Balkarian and Karachay-Cherkess Republics, Krasnoyarsk Territory, Voronezh, Penza, Saratov, Tula Regions), a system of indicators and indicators was proposed for special (correctional) educational institutions. The main issue in the formation of such a system is the definition of the goals of the changes being made and the search for indicators and indicators that track the degree of their achievement. In total, more than 80 indicators and indicators were developed and tested.

The set of indicators is developed in such a way as to reflect the actual situation in the provision of special educational services as fully as possible. Unlike indicators, target indicators set the level to be achieved. The target value of the indicator is determined based on legislative norms, and from the actual values ​​of the tested indicators.

Education of children in correctional educational institutions, as well as the creation special conditions for children with certain developmental disabilities in ordinary educational institutions - a technological process that, in the transition to the formation of budgets on a per capita basis, when a service is financed, and not a network of institutions, requires regulation and monitoring.

To effectively manage a process, clear and accurate information about its operation is required. Implementation of the key principle modern systems quality - "continuous improvement" - requires constant analysis of the current situation and making adjustments.

In many cases, the lack of relevant information is the main reason for the problems that arise. The value of information is determined by its reliability, completeness and timeliness. In order to receive useful information about the process, it is necessary to decide on the answers to such questions: what data is needed, where and how to obtain data, how to use information to make decisions?

In answering the first question, one can turn to the method of causal analysis, from which it follows that at any stage of any technological process need to collect data:

About the documentation according to which the events take place;

Personnel directly performing the work;

Material resources used;

Equipment involved;

Conditions for the implementation of the process.

The second question is the key and the most difficult. For his effective solution meticulous detail needed

Practice

processes up to a specific contractor, which will allow you to determine the answer to the question “where”. When answering the “how” question, it is necessary to take into account the specifics of the process. Obviously, obtaining reliable information is impossible without the standardization of processes. At the same time, standards should not only fully detail the process, but also clearly define the place of receipt, the method of registration and the method of summarizing information. Thus, the introduction of a per capita standard in education is one of the forms of standardization, which makes it possible to take into account both the general and the special in indicators.

Having collected the data, it is necessary to decide how to use the information for decision making. Obviously, the goal is to develop a set of measures aimed at improving processes. The collected and processed information will identify weaknesses that need to be addressed first of all.

As a result of taking into account and using the information received, decisions are made:

Should become more objective and specific, as they will be based on documented facts;

Will increase the efficiency of the continuous improvement process, as they will be aimed at eliminating the weakest points identified objectively;

Allow all staff to be involved in the quality management process.

The question of the efficiency and effectiveness of the provision of public services is perhaps the main one in the analysis of budget expenditures in general and in education in particular. Education is becoming the largest item of expenditure of the consolidated budgets of the subjects of the federation in the social sphere.

The generally accepted way of developing indicators and indicators of service delivery, widely used in developed countries, for example in England, is to distinguish three main characteristics of the result of activity: savings, efficiency and effectiveness.

Saving is the acquisition of human and material resources of the right quality and in the right quantity at the lowest cost.

Efficiency is getting the maximum result with a given set of resources or using the minimum resources to obtain the required quantity and quality of services provided.

Efficiency is the satisfaction of the needs of citizens, the solution of the tasks set, the achievement of the intended goals as a result of the implementation of the program or the performance of work. In other words, performance assesses whether the this service the goals set for it.

To ensure comparability of data, within the framework of the research work of the Center for Universal Programs,

E.N. Nikitin

Indicators and target indicators of efficiency and effectiveness of service delivery...

The indicators collected by the statistical authorities are zirovanny. The collection of primary information was carried out according to standard forms of state statistical reporting.

Special attention was given to indicators characterizing financing, the state of the material and technical base of educational institutions, the number and composition of pedagogical and medical workers, the implementation of educational programs as the main aspects of the activities of educational institutions.

As the main measure of effectiveness, the degree of accessibility (today is not a standard indicator for the education system) and the level of efficiency of the service provided were considered. Inequality in the financing of educational institutions is the first sign of inequality in access to education services. Introduction regulatory funding increases the accessibility of training through the provision of equal financial opportunities and increased transparency in funding. Thus, reducing the differentiation of spending per student is also an indicator of performance.

The target indicators developed as part of the study reflect the efficiency and effectiveness of the provision of services in educational institutions. different types and types, as well as the compliance of the level of material and technical base with the needs of children and the availability of educational programs.

The analysis of information on 7 constituent entities of the Russian Federation was carried out using the following evaluation criteria: adequacy, sufficiency and relevance of the proposed target indicators.

The adequacy of target indicators is the level of compliance of the system of target indicators created with the help of the information received with the real process of providing educational services to people with disabilities (PWD).

The sufficiency of target indicators is an indicator of performance, expressed as the ratio of the number of developed target indicators to the total number of possible target indicators. The number of possible indicators is determined by summing the developed indicators with additional indicators proposed by the regions during the pilot, and is limited to the purposes of using the indicators.

The relevance of target indicators - the importance, significance of target indicators at the present time for their use in the process of providing educational services to people with disabilities, the modernity, topicality of using target indicators.

Based on the results of approbation of the system of indicators and target indicators of the effectiveness and efficiency of providing information

Practice

educational services for people with disabilities, 4 groups of target indicators were identified in terms of their assessment according to the criteria of adequacy, sufficiency and relevance.

Group I (44% of the total number of target indicators) - statistics are kept at the level of an educational institution, for example: the share of expenses for the purchase of special educational facilities in the total amount of expenses of a special (correctional) educational institution (SCOU) by groups of diseases, the average time spent using equipped sports resources by SKOU students, etc.

These indicators are extremely relevant for educational institutions, therefore, even in the absence of statistics on them at the regional level, educational institutions independently collect information.

Group II (6%) - statistics are maintained at the regional level, for example: the coefficient of differentiation of unit costs for the wage fund (payroll) of teaching staff per student, the coefficient of differentiation of unit costs for the payroll of teaching and support staff per student, the coefficient of differentiation of unit costs for partial providing material costs directly related to the educational process, etc.

These indicators are relevant for inter-regional and inter-municipal comparisons, adequately reflect the degree of accessibility of educational services for people with disabilities.

Group III (5%) - statistics are not collected, for example: the share of vacancies in the total number of positions of teachers teaching PWDs, the share of expenditures on innovative programs for PWDs from the total costs of the institution, the percentage of PWDs who are trained in integrated educational programs in regular classes/ group-groin, etc.

These indicators are relevant and adequate in terms of monitoring the quality of the provision of educational services to PWDs in terms of the social significance of the results, they reflect the specifics of funding education for PWDs. Their absence in state statistics impoverishes the system of target indicators.

Group IV (45%) - information is collected systematically by authorities state statistics at all levels.

Based on the testing results, we found various ways definitions of indicator values:

1) target values ​​of indicators can be set by law, for example, indicators of maximum occupancy, the ratio of the payroll of teaching and non-teaching staff, etc.;

2) with low differentiation of actual values, the target indicator can be the average, maximum or minimum value for the region or municipality (depending on the target settings).

E.N. Nikitin

Indicators and target indicators of efficiency and effectiveness of service delivery...

To determine the values ​​of target indicators, calculations were made for 56 institutions in 7 constituent entities of the Russian Federation. In table. Table 1 shows some values ​​of indicators, on the basis of which target values ​​of performance indicators can be determined.

Table 1 Indicators of the effectiveness of financing the training of individuals

with disabilities, 2007 (example)

Mean value Standard deviation Coefficient of variation Minimum Maximum

Number of students per teacher, people 4 1.76 0.50 2 7

Number of students per employee of educational support staff, persons 8 3.52 0.42 3 12

Number of students per employee of administrative, managerial and service personnel, people 10 5.46 0.56 3 17

The share of the wage fund of teachers in the total payroll of the institution 0.42 0.06 0.13 0.36 0.50

The share of the wage fund of teaching and support staff in the total payroll of the institution 0.18 0.06 0.32 0.11 0.25

The coefficient of variation greater than 0.33 indicates that the corresponding values ​​of the indicator differentiation is high. In this case, the use of averaged values ​​for these indicators to determine the value of target indicators at the regional level is impossible. Here, the variant of the minimum, maximum, or other normatively established indicator value is suitable.

As a result of the work carried out, it was possible to build integrated system evaluate the effectiveness and efficiency of the provision of educational services to PWDs and give recommendations to the regions for the further use of this system.

The proposed system of indicators and indicators is based on a general classification approach used by most developed Western countries to link cost and result indicators: indicators and indicators of costs (input), product (output), result (outcome), effectiveness (effectiveness), efficiency ( efficiency, productivity). Using this classification, indicators and target indicators that can be used in combination in the field of special (correctional) education are presented in Table. 3. Such a classification of indicators was proposed by the In-

Practice

Institute of Urban Economics" based on the study of the experience of using indicators in Western countries.

Indicators of the effectiveness and efficiency of the provision of educational services provide a factual picture for analysis, and also allow the state or local authority authorities to plan the values ​​of these indicators for the medium term (about 3 years ahead) in order to achieve target indicators.

Average values ​​for some of the analyzed indicators for 2007 for 7 regions, which can be used as a starting point when planning for a three-year perspective, are given in Table. 2.

Table 2 Indicators of the effectiveness of financing the training of individuals

with disabilities, 2007

(selection by analyzed regions)

Number of students per teacher (actual), people 3.27 6.09 2.47 3.25 3.20 1.89 3.10

The share of the wage fund of teachers in the total payroll of the institution 0.50 0.35 n/a 0.42 0.48 0.71 0.46

The share of the payroll fund for teaching and support staff in the total payroll of the institution 0.16 0.20 n/a 0.17 0.27 0.05 0.20

The share of the payroll fund in the total expenses of the institution for the year 0.67 0.62 0.56 0.62 0.65 0.69 0.62

Area of ​​classrooms per student, sq. m 13.20 5.98 7.04 10.15 7.65 4.27 9.20

The area of ​​sports halls per student, sq. m 1.80 0.38 1.51 0.86 0.80 1.08 1.01

The ratio of the planned capacity of the building to the actual occupancy rate SKOU 2.13 1.37 n/a 1.07 0.98 1.13 1.10

E.N. Nikitin

Indicators and target indicators of efficiency and effectiveness of service delivery...

End of Table 2

Tula Region Karachay-Cherkess Republic Penza Region Saratov Region Krasnoyarsk Territory Kabardino-Balkarian Republic Voronezh region

Average class size in SKOU, persons 8.89 8.47 8.02 7.66 9.61 7.00 8.12

Specific expenditures per student for the educational institution as a whole, rub. per year 66,435.30 97,386.05 133,470.80 93,927.87 132,675.02 107,753.44 n/a

Specific expenditures per student with disabilities according to the payroll of teachers teaching in classes/groups for people with disabilities, rub. per year 19,627.27 21,384.94 26,250.00 24,651.43 34,601.15 52,108.23 n/a

Specific costs for the maintenance of buildings and utility costs per unit area, rub. per year 558.37 1,050.37 211.21 510.45 1,978.55 397.13 n/a

As a result of the delimitation of budgetary powers, since 2003, many norms of federal legislation that directly affect the cost of educational services provided to PWDs have been canceled (for example, a single tariff scale, salary supplements for teachers working with PWDs, etc.). All these norms influenced the target indicators of the efficiency and effectiveness of the provision of services in special (correctional) education.

Some of the federal norms have remained, but the legal acts that introduced them cannot currently be used even as recommendations in determining standard costs for regions and municipalities. These are such norms as the occupancy of classes, the ratio of rates of staff groups, etc. This also makes it difficult to determine target indicators at the regional and municipal levels.

It should be noted that the difference between indicators and indicators is that indicators set the level to be achieved. The rule to be implemented

Practice

can either be established by law, as in the case of sanitary and hygienic standards, or found in practice.

For example, the target setting might be “1 point improvement in grades” or “15% reduction in sickness absences”. Such target indicators allow not only to compare institutions among themselves, but also to meet the requirements of “achieving personal results”, which is important, since comparison of institutions among themselves can be difficult due to the lack of comparability of the initial information.

In table. 3 shows indicators and indicators that can be used as criteria for evaluating the activities for the provision of services in an educational institution.

Table 3

Criteria for evaluating the activities of an educational institution

Target indicators

Statutory standard1 Indicative standard

None developed countries

According to approximate typical states depending on the size of the school Can be established based on the normative ratio of the staffing of administrative, service and other personnel with the number of teaching staff carrying the teaching load

As a percentage of the average nominal accrued wages in a constituent entity of the Russian Federation Established by the region

None Can be installed at the municipal or regional level

Indicators (fact, plan)

Cost indicators (assessment of the resources required to provide an educational service is not lower than the guaranteed minimum)

Provision of schools with modern materials and equipment

Ratio - student/teacher ratio

Number of staff involved in administration, building maintenance and other non-teaching work

Average nominal accrued wage teachers of educational institutions

The share of teachers in educational institutions who have undergone advanced training and retraining in the current year, in their total number

1 Federal regulations.

E.N. Nikitin

Indicators and target indicators of efficiency and effectiveness of service delivery...

Continuation of the table. 3

Target indicators

Product indicators (volume of services provided) Average class size at the 1st, 2nd, 3rd levels of education 5-12 people Possibility of focusing on lower occupancy in small rural schools

Percentage of children studying above the state standard (in extracurricular activities, in circles) Absent Can be set at the municipal or regional level

Number of circle work hours Determined based on the norms of federal legislation: the size of the school, the teaching load, the rates provided for by the standard states It is possible to increase the number of circle work hours at the expense of additional budgetary and non-budgetary funds

Coverage of children aged 5-18 years with programs of special (correctional) education None Determined at the regional level

The proportion of children aged 5-18 with disabilities receiving education in special (correctional) educational institutions, in their total number Absent Can be established at the municipal or regional level

Outcome Indicators (Benefits from Service Delivery) Percentage of Standard Funding Applications Approved by Schools N/A Can be set at municipal or regional level

Percentage of graduates who continued their studies Absent Can be set at the municipal or regional level

The share of graduates of educational institutions who passed the final certification in USE form, v total strength graduates None Can be installed at the municipal or regional level

Practice

The end of the table. 3

Target indicators

Indicators (fact, plan) Statutory norm Indicative norm

Outcome indicators - Percentage of exceeding the number According to SanPiN norms Orientation to federal norms

activity (the degree of pupils in relation to the area of ​​educational

undermining public opportunities for schools

significant goals - ka - 3.5 sq. m

accessibility and free of charge) Percentage of school lessons, None Reasons for implementation

not carried out leniya "aktirovaniya" regulations

during the year for a reason are mented by local

cancellation of classes due to and regional laws

with unfavorable weather conditions

Attendance rate None Comparison with municipality, region, country averages

Performance indicators Share financial resources, None Comparison with average

(costs not transferred to the school by sub-indicators for municipalities)

used on a semi-colour basis for a palette, region, country

Result) Costs for centralization None Comparison with average

new administration in terms of the region,

per student country

Share of maintenance costs None Comparison with average

building and repair indicators for municipal

in the structure of expenses of an educational institution by palette, region, country

Specific costs for special services None Comparison with average

social (correctional) indicators for municipal

education per unit palette, region, country

population

Total capital value None Comparison with average

resources per year in the calculation of indicators for the municipal

those per student per palette, region, country

Total value per capita None (reco- Determined by region

costs mental characteristic and municipal

ter for the purpose of calculating educational subventions) level

Building cleaning costs Approximate norms Reference to federal norms

per unit of plotype states, the Sanral Legislation

spare piN wa and the financial capacity of the municipality

Maintenance costs Absent Determined by the municipality

buildings per unit area (electricity, water supply, heating, etc.) floor level

E.N. Nikitin

Indicators and target indicators of efficiency and effectiveness of service delivery...

As an example, let us evaluate the values ​​of some indicators and indicators of the provision of an accessible free educational service in special (correctional) educational institutions at a level not lower than the constitutionally guaranteed minimum.

Suppose the actual value of the student/teacher indicator in a special (correctional) educational institution is 6 students per teacher. This indicator can be assessed using a target indicator set as a benchmark at the regional or municipal level. If the indicator is 5 students per teacher, then the value of the actual indicator satisfies the maximum need of the region and (or) the municipality in teaching staff, taking into account the level of quality and efficiency of the provision of budgetary services set by the region. The value of the actual indicator 6: 1 in this particular case indicates a higher level of efficiency in the use of budgetary resources, since the number of teachers does not exceed the maximum requirement for the implementation of the program of special education in full.

Another example. Let's say the share of the payroll for staff involved in administration, building maintenance and other non-teaching work is 31%. Since there is no federal legislation strictly regulating this norm, it should be established by the region or municipality itself, the administration of which can determine the share of the payroll of non-pedagogical personnel in the wage fund of educational institutions, both based on the existing ratio, and based on their own ideas about the effective spending of limited local funds. budget. Based on exemplary typical states, this value can be estimated1 at the level of 30-35%. This category does not include teaching and support staff.

The average occupancy of classes can be compared with the value established in the Model Regulation on a special (correctional) educational institution - this is 5-12 people. The occupancy rate below 5 people indicates the inefficient provision of educational services, it can be considered as a signal for the enlargement of classes when laying in the norm of 5-12 people, as in the Model Provision. If it turns out that the consolidation of classes for one reason or another is impossible, the norm may be reduced. In any case, the normative size of the occupancy of correctional classes must be approved by the region and (or) the municipality.

According to the norms of SanPiN, the area of ​​​​the classroom for one child should be 3.5 square meters. m. If the actual value is lower, then it can be argued that in a special (correctional)

1 Expert judgment.

Practice

In an educational institution, there is a shortage of classrooms: the total number of children exceeds the number of places in classes that meet SanPiN standards.

The amount of specific spending on special (correctional) education per student can be compared with such target indicators as the average specific spending on special (correctional) education in the Russian Federation in a group of regions with the same socio-economic situation or the average specific spending on general education for basic general education programs in ordinary classes of general education schools, multiplied by the coefficient of appreciation in the cost of education for special (correctional) programs in primary and basic general education schools.

Thus, using target indicators and indicators of the efficiency and effectiveness of the provision of educational services to people with disabilities, it is possible to control the level of efficiency of budget expenditures in the implementation of the educational policy of the city, district, region. This will make it possible to timely detect weaknesses in the financing of standard educational services, control the availability of quality education, as well as the budgetary security of remedial education.

To make decisions when building educational policy, the region and the municipality create their own system of target indicators, taking into account the norms approved by the regulatory legal acts of a higher level of government. The main task here is that each introduced norm at the regional or local level must be supported by a legal document.

High-quality and appropriate education of people with disabilities is considered as a priority contemporary politics in education. In this regard, there is a need to analyze the problem situation related to training, integration, social adaptation of people with disabilities, to determine modern principles and organizational approaches to its solution. Therefore, the evaluation through the system of target indicators of the effectiveness and efficiency of the provision of educational services in the framework of the implementation of various educational programs for children with disabilities is of paramount importance.

In the constituent entities of the Russian Federation, a large amount of information is currently being collected regarding certain aspects of the education system. Information is received by both statistical authorities and education authorities at the regional and local levels. Most of the information is collected on educational institutions, but a comprehensive assessment of the activities of the institution (from the amount of funding to the final results) cannot be obtained. Therefore, it is practically impossible without special

E.N. Nikitin

Indicators and target indicators of efficiency and effectiveness of service delivery...

The purpose of the study is to evaluate the relationship between the material and technical equipment of the institution, the qualifications of teachers and other indicators of the availability of resources with the final result, and, as a result, with the quality of education.

Indicators and target indicators of the efficiency and effectiveness of the provision of educational services to people with disabilities are needed for each level of education, and these are not only statistics, but also information for internal control, to make managerial decisions that help improve the current situation.

Based on the results of testing the proposed system of target indicators and indicators of the effectiveness and efficiency of the provision of educational services to people with disabilities, we can draw the following conclusions:

On the possibility of their application not only at the level of a constituent entity of the Russian Federation, but also at the municipal level - if the founder of a special (correctional) institution is a municipality - and at the level of an educational institution;

Opportunities to integrate performance indicators in order to conduct inter-regional comparisons at the Federation level;

Possibilities of building educational policy in the system of correctional education based on the analysis of target indicators and performance indicators in dynamics;

The possibilities of applying the considered approach not only in the field of special (correctional) education, but also in the education industry as a whole, including the federal level of decision-making, as well as in other sectors of the social sphere in which budgetary services are provided to the population.

1. Model regulation on a special (correctional) educational institution.

2. Materials of the Audit Commission & Improvement and Development Agency (IDeA) "BVPP Local Performance Indicators", September 2001.

3. Garadzha M.Yu. Monitoring the effectiveness of budget expenditures: the main stages of implementation at the local level. M.: Fund "Institute of Urban Economics", 2002.

4. Mechanisms for combining normative budget financing and financing, taking into account the quality of work of educational institutions / ed. N.G. Tipenko. M., 2005.

5. Performance-oriented budgeting: international experience and opportunities for application in Russia: Dokl. Center for Fiscal Policy. Moscow: Academia, 2002.

6. Poletaev A.V., Agranovich M.L., Zharova L.N. Russian education in the context of international indicators: Comparative report. Moscow: Aspect-Press, 2003.

7. Agranovich M.L., Zharova L.N., Kozhevnikova O.N. Education systems of Russian regions: Comparative analysis based on international indicators. Moscow: Aspect-Press, 2004.

Literature

Practice

8. Target indicators and performance indicators of the mechanism of normative per capita financing of persons with disabilities. Materials of the final seminar-meeting / ed. N.G. Tipenko. M, 2006.

9. Development and approbation of a system of target indicators and indicators of the effectiveness of the transition to the mechanism of normative per capita financing of education for people with disabilities, taking into account the quality of education for institutions of all levels of education. Materials of the report on the project, prepared by ANO "Center for Social Research and Innovation" together with LLC "Center for Universal Programs" by order federal agency in education in 2007

Already in 2002, the Interdepartmental Program for the Development of the DL System, approved by Order of the Ministry of Education of the Russian Federation No. considers the development of children's preschool education as one of the priority areas of educational policy. Already in the model regulation on UDO, it is emphasized that the Institution is responsible for the quality of the programs implemented; compliance of forms, methods and means of organizing the educational process with the age, interests and needs of children.

The study, detailed analysis of the legal documents regulating the educational process in the DL system shows that the DL system has a legal basis for ensuring the activities of the DLOD and prerequisites for developing criteria, indicators and conditions for the effectiveness of the implementation of additional educational programs of various directions. But only preconditions...

And the beginning was laid already in 1997 at the scientific-practical conference in Yaroslavl "The problem of the result and quality of the activities of the UDOD". In 1997, the collection “Problems of the effectiveness of pedagogical activity” was published in the Herzen RSPU, scientists O.E. Lebedev, E.I. Kazakova, R.U. Bogdanova, E.V. Titova, A.I. Osminina, A.P. .Tryapitsyna revealed various facets of this problem in their research. But today there are no unified (generally accepted) scientific definitions that reflect the essence of such concepts as result, effectiveness, efficiency, and even more so approved methods. Thus, the problem of results and effectiveness in distance education is in the center of attention to this day.

Control, or verification of learning outcomes, is an essential component of the learning process. Control allows you to determine the effectiveness of training under the program, helps children, parents, teachers to see the results of their work, which creates a good psychological climate in the team and increases the self-esteem of the student. Unlike general education, where the process of identifying the results of students' educational activities is quite clearly defined by state standards, in additional education for children this issue causes real difficulties for teachers.

The urgency of the problem is also due to the process of state certification and accreditation of the UOD, when an "external" assessment of the performance of institutions is given. Here there is a contradiction between the possibilities of additional education for a comprehensive solution of the problems of education, upbringing and development of children and the formalized nature of the performance indicators in the UOD (as a rule, institutions take into account only one aspect of performance - students' achievements in reviews, competitions and festivals). In addition, there are currently no clear performance criteria as a multi-functional, multi-aspect, multi-level system.

In this article, I want to summarize, summarize and present to you the results of a study by the City Center for the Development of Additional Education, and a study of various categories of UDOD specialists (methodologists, teachers, representatives of the administration), which were already carried out in 1999 and the practical experience of the Department of Additional Education of the St. Petersburg Suvorov Military School of the Ministry of Defense of the Russian Federation. My managerial and teaching activities in these institutions constantly faced the problem of the effectiveness of educational programs in additional education. The solution to this problem is presented in this article.

An analysis of the scientific psychological and pedagogical literature shows that at present there is no single interpretation of the concepts of "result" and "performance" among scientists. However, there are 3 main approaches to the definition of these concepts.

First of all, the result is the result - of the activity, process, interaction of its main participants (T.V. Ilyina, A.V. Zolotareva, L.G. Loginova). Another approach is to define the result as changes - states, properties of subjects or objects, an individual or cumulative product or process, a system of relations, etc. (R.M. Gainutdinov, I.V. Shinkevich, T. Pshcholovsky). And, finally, the third approach is to consider the result as a state - a system, things, fixed at a certain point in time (V.A. Kartashov, T. Pshcholovsky, R.U. Bogdanova).

Compliance with the goal;

The consequence of any organized action;

Intermediate - final;

Individual - cumulative;

External (visible) - internal (personal);

Predictable (planned) - unpredictable (unforeseen consequences);

Positive - negative;

Actual (received immediately) - delayed (manifested after a certain time).

Thus, the result of the educational process in distance learning should be considered as objective changes that manifest themselves at the levels:

The personality of the child and the teacher,

relationship systems,

- "products" of pedagogical activity and the activity of the child in the educational process.

For most teachers, the most common is to track the results of participation in contests, competitions, reviews. How to fix these results? Here is one example:

Information card of the results of children's participation in competitions, festivals and competitions of various levels

This technique is used to record and evaluate the results of children's participation in competitions, festivals, competitions of various levels, that is, to evaluate "external" achievements.

The following forms of "external" presentation of students' achievements are distinguished:

competitions, reviews, festivals, exhibitions, competitions, conferences, seminars, round tables, readings, olympiads, concerts and much more.

When filling out the card, it is advisable to indicate the names and dates of the events in which the child took part. Achievements are recorded at the following levels: team, institution, city (district), international and Russian. Qualitative indicators of the result are also determined: participation, prizes, diplomas, winner. Each indicator, depending on the degree of significance, corresponds to a certain score.

A variant of the information card form (from the experience of the Suvorov Military School)

Surname, name of the student __________________________________________________

Age ________________________________________________________________

Name of the team _____________________________________________________

Year of study ___________________________________________________________

Date of filling out the card _________________________________________________

Data processing and interpretation.

In accordance with the results of the child's participation in competitions and festivals of various levels during the school year, the points are entered into the card and added up. The sum of points determines the rating of students, both in the study group and in the children's team as a whole, according to the parameter "external" effectiveness of educational achievements. The methodology can be carried out both on the basis of an expert assessment (by a teacher) and on the basis of a student's self-assessment.

It is advisable to identify and analyze these results 2 times a year: based on the results of the first half of the year and the year. With regular implementation, the methodology allows you to fix the dynamics of students' achievements, as well as stimulate their creative activity.

The following methods for determining the effectiveness of the implementation of educational programs for additional education of children, developed by N.V. Klenova, L.N. A. I. Herzen and GOU "SPb GDTYu", tested in pedagogical practice and are a tool for assessing the competence of pupils.

This material will allow teachers of additional education for children to correctly approach the choice of methods for evaluating the effectiveness of an educational program, find and develop their own methods and improve the quality of their program implementation and, most importantly, improve the quality of education.

Methodology "Portfolio"

("Briefcase")

One of modern forms assessment of achievements and competence, including the student's creative success, is the formation of a "portfolio" (Portfolio). Maintaining a portfolio develops students' skills of reflective activity (the ability to analyze their own activities, improve them, take the initiative to achieve success). The content and methods of designing a "portfolio" can be very diverse - from a complete collection of all works to an album of the highest achievements.

Currently, the following types of portfolios are distinguished:

§ Portfolio of documents - the most common form, which is a file folder with a variety of information about the experience of successful extracurricular work acquired by the student over a certain period of time. The list of documents includes report cards, certificates, diplomas, feedback letters, certificates of participation in various events.

§ Portfolio of work reflects the efforts, progress and achievements of students in a particular area of ​​activity. This is a collection of various creative, design, research works of the student, as well as a description of the forms and directions of his creative activity - participation in competitions, achievements, etc.

§ Portfolio of reviews includes notes of the teacher about the work, grades for completed tasks with characteristics and explanations, as well as a written analysis of the student himself of his specific activity and its results. The portfolio can be presented in the form of texts of conclusions, reviews, summaries.

The most appropriate is a comprehensive portfolio option, which includes a section of documents, a section of works and a section of reviews.

There is no clear list of items and the number of items that must be included in the portfolio. This issue is decided by a specific teacher, a group of teachers or a methodological association.

Methodology "Pedagogical diary"

This form of diagnostics can be used by teachers working with individual learning groups (solo singing, instrumental classes, etc.). The pedagogical diary is a document in which the teacher, by observing and analyzing individual characteristics of the student (for example, the level of creative development, the development of interests), studies the individual dynamics of the development of these qualities.

Sections of the diary can take into account the specifics of a particular area of ​​activity. The diary of observations consists of 4 sections:

1. "Initial data": a brief description of the student at the beginning of the development of the educational program, assessment of the level of achievements in the chosen type of activity, character traits, health, creative interests.

2. "Achievements in the subject area": ​​twice a year, the achievements of students are recorded.

3. "Creativity in self-development": expansion of interests, self-education efforts.

4. "Conclusions and plans": the results of the work for the year and a plan to support the student in the future.

Methodology "Protection of abstracts"

One of the common methods of public presentation of the results obtained by the student is the defense of abstracts in a scientific and practical conference. Especially if the activities of students during the development additional programs general cultural level is more connected with the theoretical material. This technique is aimed at a comprehensive assessment of both the subject component of the child's activity and over-subject (in particular, communicative) skills.

An abstract is an independent work (project) that requires the student to analyze and summarize information independently found in various sources.

This technique requires a clear organization of both the preparation process and the actual defense of the abstract.

Stages of implementation of the methodology:

1. Approval of the topic of abstracts. Organizing this stage, it is necessary to take into account the individual characteristics of students (degree of creativity). The best option is to offer a specific set of ready-made themes with the possibility of offering your own theme. In this case, individual students may need help in choosing a topic, but it is important that the final choice is left to the child. The most enthusiastic and creative students, as a rule, offer their essay topics. For them, there should be a mandatory procedure for approving the topic (preferably not by one teacher, but collectively), first of all, in order to maintain a certain level of depth and volume of research in all children's research.

2. Guiding the preparation of the abstract. At this stage, the individual characteristics of children should also be taken into account by varying the level of participation of the leader (consultant) in the study of a particular pupil. The readiness of the student to listen to the opinion of the consultant and take it into account is one of the indicators of the effectiveness of mastering the general cultural educational program. Such behavior testifies to the student's understanding of the traditional approach to the organization of scientific and technical work at the stage of theoretical study of an objective reflection by him of his level of subject competence. but a large number of calls to a consultant can also indicate the opposite: a serious lack of specific knowledge, so here you should pay attention, first of all, to the quality of the questions asked and the frequency of requests for help.

3. Reviewing the abstract by a specialist in the relevant field, who evaluates the work according to several indicators: the depth of the topic, the correct use of terminology, the quantity and quality of the sources of information used, the validity of the conclusions, the design of the work.

4. Protection of the abstract. At this stage, the main assessment of the success of the student in mastering the educational program takes place. Each student makes a report (8 - 10 minutes) on the main content of the abstract using visual means of presenting information: posters, handouts. The report may be accompanied by a computer presentation. Questions to the speaker are asked not only by teachers, but also by other guys. On defense, in addition to the content side of the essay, the student's ability to summarize his own work when compiling a report, freedom of ownership of the topic, the ability to answer a question in his own words, the ability to quote from the text, self-confidence and respect for opponents are assessed.

An important aspect of the defense of the abstract is the question to the student about the possible options for continuing the work he has begun - the proposed topics for other studies. The answer to this question may allow us to assess the breadth of the student's outlook, his awareness of the existing directions in the field of activity being studied - important characteristics of the development of additional educational programs of the general cultural level.

Summarizing all the methods, it can be stated that monitoring the results of a child's learning in an additional educational program should be carried out according to two groups of indicators, since educational activities in the system of additional education involve not only teaching children certain knowledge, skills and abilities, but also developing the diverse personal qualities of students, this such indicators;

Educational (fixing subject and general educational knowledge, skills, acquired by the child in the process of mastering the educational program;

§ Personal (expressing changes in the child's personal qualities under the influence of classes in a given circle, studio, section).

The data can be presented in the form of a table that allows you to clearly demonstrate the set of basic knowledge, skills and practical skills that a child should acquire as a result of mastering a specific educational program (using the example of St. Petersburg Higher Educational School).

The technology for determining learning outcomes for an additional educational program is as follows: the set of measured indicators (theoretical, practical training of the child, general educational skills and abilities) is evaluated by severity (from minimum to maximum). For convenience, the selected levels are indicated by the corresponding test scores (1 - 10 points). The methods by which the teacher will determine the compliance of the child's learning outcomes with program requirements can be observation, testing, a control survey (oral or written), analysis of a control task, an interview, etc. This list of methods can be supplemented depending on the profile and specific content of the educational program.

The dynamics of the results of mastering the subject activity of a particular child is reflected in the individual card for recording learning outcomes for an additional educational program (Table 2). The teacher twice a year (at the beginning and at the end of the school year) puts down points corresponding to the degree of severity of the assessed quality in the child. In addition, at the end of the card, the teacher is invited to highlight a special column “Subject Achievements of the Student”, which plays the role of a “portfolio”, where the most significant achievements of the child in the field of activity studied by the educational program are recorded. Here the results of the child's participation in exhibitions, olympiads, contests, competitions, etc. can be noted.

Regular monitoring of the results can become the basis for stimulating, rewarding the child for his work, diligence. Each assessment needs to be commented on, to show what the increase in the knowledge and skill of the child is - this will support his desire for new successes.

You can advise the child to keep a record of his own educational achievements (Table 3). To do this, he is recommended to have a special notebook (diary) and gradually fill it out. It is necessary to teach children to reason about the quality of their work: this is of great importance for the formation of children's self-esteem.

Self-assessment allows children to record their own progress through the levels of mastery. If it is done openly, then social mechanisms are included in its regulation. An open display of learning outcomes under the program stimulates children to look for new job options, to creative activity.

For a child, the assessment of his work by parents is of great importance, so the teacher needs to think over a system of working with parents. In particular, control activities can be combined with parent meetings so that parents can see the growth of their child during the year based on the final work.

Table 1

Monitoring the results of a child's learning in an additional educational program

table 2

Individual card

Surname, name of the child __________________________________________________________

Age_____________________________________________________________________

Type and name of the children's association ___________________________________________

Full name of the teacher _____________________________________________________________

Start date of observation _________________________________________________________

Table 3

Self-Assessment Scheme

An example of another method for determining the knowledge gained by a pupil

any association:

Map of self-assessment by the student and expert assessment by the teacher of the student's competence

This technique is intended for diagnosing the results of mastering the educational program by students. In addition, the methodology contributes to teaching the child to assess the level of competencies achieved (theoretical knowledge, practical experience, creativity and cooperation), allows the teacher to monitor the formation of the student's self-assessment skill.

The questionnaire is intended for students aged 12-16.

The technique is carried out in 2 stages. At the first stage, students are invited to mark the level of certain competencies acquired in the process of mastering the program on a five-point scale. To do this, the student crosses out in the upper column the number corresponding to the assessment that he is ready to give himself. At the second stage, the teacher in the lower column notes his assessment of the level of student achievement.

Questionnaire form

Dear friend!

Please evaluate on a five-point scale the knowledge and skills that you received while studying in a circle (team) this academic year and cross out the corresponding number (1 is the lowest score, 5 is the highest).

Processing of questionnaires and interpretation of results.

The student's self-assessment and the teacher's expert assessments are summed up, the arithmetic mean value is calculated for each indicator, and then on mastering the program as a whole.

Such a logic of conducting a survey allows not only to determine the level of competence of students, but also to identify the features of their self-esteem based on comparing the opinions of children with the opinion of a teacher.

Monitoring the personal development of the child

in the process of mastering an additional educational program

The development of the child's personal qualities should be provided for in each educational program.

The development of a child's personality is influenced by many factors, and not just communication with a teacher of additional education. It is very important to find those indicators of personal development, on the basis of which it is possible to determine their positive dynamics.

So, when studying changes in a child's personality, one can focus on such performance parameters as motivation, "I-concept", value orientations, competence.

To diagnose these parameters, the following methods were used:

- "Educational Needs" and "Map of Interests",

Fiedler and Ketell tests,

Methodology "Value Orientations",

Cards of self-assessment by students and expert assessment by the teacher of the student's competence in mastering the experience of theoretical information and methods of practical activity, acquiring creative experience.

As an example, I want to give one of the methods that involves tracking the following personal qualities: organizational-volitional, orientational, behavioral qualities of a person (Table 4). Taken together, the personal properties given in the table reflect the multidimensionality of the personality; allow you to identify the main individual characteristics of the child, are easily observed and controlled, available for analysis by any teacher and do not require the involvement of other specialists.

The technology for determining the personal qualities of a student is as follows: the totality of measured indicators (patience, will, self-control, self-esteem, interest in studies, conflict, type of cooperation) is evaluated according to the degree of severity (from minimum to maximum). For convenience, the selected levels are indicated by points. Observation, questioning, testing, diagnostic conversation, reflection method, incomplete sentence method, and others can be used as methods for diagnosing personality changes in a child.

The technology of monitoring the personal development of a child requires documentation results for each child. For this purpose, the teacher draws up for each child an individual card for recording the dynamics of the personal qualities of the child's development.

The card is filled out twice a year - at the beginning and at the end of the academic year. If necessary, this can be done more often, for which additional columns can be introduced.

The resulting sections allow you to consistently record the step-by-step process of changing the personality of each child, as well as plan the pace of individual development.

The student himself can be involved in the evaluation of the personal qualities listed in the card. This will allow, firstly, to correlate his opinion about himself with those ideas of the people around him; secondly, to clearly show the child what reserves he has for self-improvement.

Table 4

Monitoring the personal development of a child in the process of mastering an additional educational program

Individual card

taking into account learning outcomes for an additional educational program

(in points corresponding to the severity of the quality being measured)

Surname, name of the pupil ________________________

Age______________________________________________________

Type and name of the children's association ______________________________

Full name of the teacher __________________________________________________

Start date of observation ________________________________________________

* The IV block can be entered into the card at the discretion of the teacher in order to mark the child's special successes in conscious work on changing their own personal qualities.

It is very important that the pupils fulfill the required educational products, their complexity is also regulated by the educational program in accordance with the period of study. For example, according to the results of the first year of training, this is one result, according to the results of the third year of training, it is more complicated with certain details in accordance with the specifics of the product.

Analysis of activity products is a comprehensive analysis, thorough study, generalization and systematization of the data obtained. The teacher develops a number of criteria by which the product of the activity is evaluated (publication, photograph, work on applied art, presentation, etc.): individual or group work, long-term or short-term, etc.

Personal achievements of students, for example, in physical culture and sports, are regulated by certain standards for a particular sport.

Map of interests

This technique is a modification of the "Map of Interests" developed by A.I. Savenkov to identify the range of interests of the child. However, unlike the author, who based the classification of interests on the system of social relations (man-nature, man-art, man-man, etc.), the authors focused on the directions of additional education (artistic, technical, etc. .). This technique (Barysheva T.A., Senicheva I.O.) makes it possible to determine not only the range of interests, but also their correspondence to the orientation of students in DO.

To conduct a survey, it is necessary that each student has an individual questionnaire form. Before starting the procedure, the teacher or psychologist explains to the children why the survey is being conducted and the rules for filling out the questionnaires.

Form options

Questionnaire for students aged 6-11

Dear friend!

Read the statements below carefully and mark your choice with any icon.

Do you like? Do you like it? Would you like?

Please write:

1. Last name, first name _________________________________________________
2. How old are you__________________________________________
3. What association are you in? ______________________
4. How many years have you been with this association? __________________

All the questions raised correspond to the six areas approved in the DO: artistic, technical, physical culture and sports, tourism and local history, social and pedagogical, natural sciences.

When processing questionnaires, it is advisable to use special individual forms that clearly allow you to determine the range and profile of each student's interests, to identify their preferences.

interests

________________________________________________

(surname, name of pupil, age)

______________________________________________________

(direction, team, year of study)

By summing up the choices of students for each question of the questionnaire, it is possible to identify the common interests and inclinations of the study group, the entire children's team.

As the team of the study group is formed, the students master the program, the interests of the children may change. Therefore, it is advisable to conduct a survey annually, preferably at the beginning of the academic year.

The data obtained (both individual and group) can become the basis for the teacher to determine the individual educational route of the student, the choice of appropriate forms and technologies for conducting classes, educational work, organizing effective interaction in the "teacher-student", "student-student" systems .

Methodology "Value Orientations"

This methodology, developed on the basis of the methods of studying the value orientations of M. Rokeach and L.A. Yasyukova, provides for the study of the orientation of the personality of the child involved in the institution of additional education for children. The methodology makes it possible to identify a system of significant values ​​that determine the most common guidelines for the life of students: the child's attitude to the world around him, to himself, which are considered as values-goals.

As practice shows, life values ​​are determined by a person consciously only in adolescence. Therefore, this technique is advisable to carry out for students aged 12-16.

The method is based on the method of direct ranking of the list of values ​​(a list of 15 values), when the child is asked to put the most significant life values ​​in the first place, and the least significant in the last place.

The survey is conducted anonymously. However, in order to be able to compare the results of several sections, it is advisable to authorize the questionnaire (for example, invite the student to indicate the last 4 digits of his phone number).

Application form option

Dear friend!

Each person chooses for himself the most important life values ​​and strives for them. Please choose from the following characteristics of life values ​​that are important to you, and number them in order of importance from 1 to 15 according to your preferences.

Data processing and interpretation.

When processing questionnaires, students' answers are grouped into categories of life values, and their arithmetic mean value is calculated:

1. Material values ​​- answer options (1+8)/2;
2. Personal moral and volitional values ​​- (2 + 11 + 12) / 3;
3. Personal family and household values ​​- (3 + 5 + 6) / 3;
4. Values ​​of knowledge and creativity - (7 + 9) / 2;
5. Spiritual and cultural values ​​- (4 + 10 + 14) / 3;
6. Communicative values ​​- (13 + 15) / 2.

The obtained values ​​make it possible to build a hierarchy (rating) of the pupil's value orientations. This can become for the teacher the basis for developing an individual educational route for the student, taking into account the most significant goals for him. At the same time, it should be borne in mind that the most important pedagogical task is not only taking into account the value orientations of the child, but also, if necessary, their development.

In addition, the implementation of the "Value Orientations" methodology in the children's team makes it possible to determine the general trends and specifics of the values-goals of a group of students, which will allow the teacher to adjust the content, forms and technologies of educational work.

Questioning of students in order to identify the rating of their value orientations is recommended to be carried out 1 time at the beginning of the school year, or 2 times - at the beginning and, as a control cut, at the end of the school year.

The question often arises, how often should pupils be assessed?

At each lesson, the teacher evaluates the student according to the following indicators:

the quality of the task;

the level of independence in the course of the task;

the degree of complexity of the task being performed;

manifestation of activity and creative abilities;

the recorded data is put down in a personal journal (according to a 5 point system).

And once a month, the teacher puts in the table the points corresponding to the degree of severity of the assessed quality in the student. In addition, at the end of the card, the teacher is invited to highlight a special column “Subject achievements of students”, which plays the role of a “portfolio”, where the most significant achievements of students in the field of activity of the educational program under study are recorded. Here the results of participation in exhibitions, olympiads, competitions, competitions, etc. can be noted.

In order to have a more holistic view of the system for monitoring the results of mastering additional programs by pupils, we bring to your attention the algorithm of this system:

· definition of criteria for individual evaluation of the pupil's activity in the creative association of DO;

· a system for evaluating the individual achievements of pupils and the work of creative associations in general;

Analysis of individual and group results achieved;

making decisions to improve the system of work in the association and their implementation;

monitoring the effectiveness of decisions made and their adjustment;

obtaining the expected result. (and everything repeats again)

The built-in monitoring system makes it possible to manage the process of harmonious and dynamic development of the student's personality and the entire system of additional education as a whole

Approbation of the proposed system for diagnosing the results of the educational process revealed certain difficulties:

Ø the need for additional financial investments to provide teachers and students with test materials,

Ø the presence of psychologists or specially trained teachers in the staff of the institution for conducting and competently interpreting the results of psychological diagnostics.

Ø significant time costs for teachers associated with identifying, fixing and analyzing the results of each student.

It is important that today every teacher and methodological services are given the opportunity to choose for practical use the material that is most important to them and meets the goals set in the educational programs.

This article discusses issues related to various aspects of assessing the effectiveness and efficiency of educational programs implemented in a network form. Various definitions of the concept of "efficiency" are considered, an analysis of various expert-analytical methods for evaluating efficiency and effectiveness is carried out. The most preferable is the construction of a balanced scorecard, which allows for a comprehensive assessment of each network educational program, evaluate it for different stakeholders, determine the impact on financial indicators activities of the organization, customer satisfaction, the quality of graduate training. T. Saaty's method of analysis of hierarchies was used as a tool in the formation of a system of indicators. The article introduces two hierarchical systems of performance and efficiency indicators, criteria and scales for each indicator. Two decision rules are proposed to determine whether a particular network program can be recognized as effective and efficient.

decision rule

hierarchy analysis method

balanced scorecard

effectiveness

'efficiency

network form of educational programs implementation

1. Badeeva E.A. University planning within the process approach [Electronic resource]. – Access mode: http://www.rae.ru/monographs/169 (date of access: 11/12/2013).

2. Gavrilov S.I., Guseva A.I., Vorontsova A.N. Tools for assessing the quality of information and educational resources / / Audit and financial analysis, 2010. - No. 6. - pp. 353-358

3. Kireev V.S. The method of Saaty hierarchies in the system for assessing the innovativeness of educational projects // Software products and systems, 2011. - No. 4. - P. 188-190

4. Decree of the Government of Russia dated March 16, 2013 No. 211 "On measures of state support for leading universities of the Russian Federation in order to increase their competitiveness among the world's leading scientific and educational centers" [Electronic resource]. - Access mode: http://government. ru/docs/818 (date of access: 11/12/2013).

5. Competitiveness Program of the National Research Nuclear University "MEPhI" [Electronic resource]. – Access mode: http://www.mephi.ru/about/competitiveness (date of access: 11/12/2013).

6. Saati T.L. Decision making under dependencies and feedback. Analytical networks / Translated from English - M .: Book house "Librokom", 2009.

7. the federal law of the Russian Federation dated December 29, 2012 N 273-FZ "On Education in the Russian Federation // Russian newspaper, December 31, 2012, federal issue No. 5976

Introduction

Transition to new version The Law on Education expands the opportunities for the competitiveness of domestic higher education institutions. Teaching students on joint educational programs with other foreign universities, the development of international academic mobility for students and university teachers becomes available due to the introduction of a network form of educational programs.

When using such a new form, the question always arises of how to evaluate the effectiveness of an online educational program.

The National Research Nuclear University MEPhI (NRNU MEPhI), as the university that won the open competition for state support to increase its competitiveness and join the top 100 universities in the world by 2020, has begun to actively form a regulatory and methodological framework for the implementation of educational programs in the network.

In this regard, the development of a methodology for identifying the presence and evaluating the effectiveness and efficiency of the application of network forms for the implementation of educational programs is particularly relevant from the point of view of increasing the competitiveness not only of NRNU MEPhI, but of all domestic higher education.

The concepts of "efficiency" and "effectiveness"

In modern literature, there is a difference between the concepts of "efficiency" and "effectiveness". In the very general view these differences are formulated as follows:

• performance reflects the degree of achievement of the planned result;
• efficiency shows the ratio of the results obtained to the time, financial and other resources spent to achieve them.

As a rule, expert-analytical methods are used to evaluate such concepts. In this case, analytically, based on system analysis, a set of indicators is identified that can be used to evaluate any property, a system of evaluation scales and the evaluation criteria themselves are developed.

By expert means, there is an agreed assessment of the significance (weight) of each indicator, the threshold acceptable values ​​for the indicators are determined, and the final objective function for the assessment is formed.

Analysis, selection and justification of expert-analytical methods for assessing the effectiveness and efficiency of the use of network forms for the implementation of educational programs

Evaluation of the effectiveness of the educational program includes an analysis of the achievement of the planned values ​​of the indicators of the educational process. For this, a quantitative assessment methodology is used, which includes:

• assessment of the effectiveness of the implementation of the work plan on the quality of the program implementation.
• assessment of the effectiveness of achieving the goals of the program.

Evaluation of the effectiveness of achieving the goals of the educational program will determine the actions necessary for planning improvements, corrective or preventive actions to improve the quality of educational services and achieve goals.

If the evaluation of the effectiveness of the educational program is based on a set of indicators, the value of which is evaluated on a binary scale, then the effectiveness of the implementation of the work plan for the implementation of the network educational program can be evaluated by the formula:

n is the number of target indicators;

xi - planned value of the i-th target indicator;

xacti - actual quantitative assessment of the i-th target indicator.

More difficult is the issue of performance evaluation. Let's try to explore the concept and existing expert-analytical methods for evaluating efficiency. In table. 1 shows various definitions of the concept of efficiency

Tab. 1. Definition of the concept of efficiency

Thus, efficiency is understood as a characteristic of the quality of the system in terms of the ratio of costs and results of its functioning.

Let's consider the most commonly used methods for evaluating the effectiveness presented in, and the possibility of their further use in evaluating the network form of implementing educational programs.

1. Evaluation of the effectiveness of the program can be carried out on the basis of the benefits from its implementation, certification and operation. In this case, a qualitative comparison of the planned and achieved results is carried out, and the study is based on the assessments of university representatives, which are subjective and can be deliberately distorted.

2. Determining the effectiveness of the program is possible on the basis of establishing a causal relationship between the implementation of the program and economic indicators organization's activities. In this case, economic efficiency can be defined as the difference between results and costs, or as a ratio between these values. Another method is the ability to calculate efficiency as the difference between the costs before and after the implementation of an activity or set of activities. In any case, this approach is limited, since the receipt of economic benefits is not the main activity for public educational institutions.

3. The third group of methods makes it possible to evaluate the effectiveness of the educational program based on the consumer satisfaction index. Within the framework of such a study, an empirical relationship is revealed between customer satisfaction and the economic performance of the organization. In this case, data on the satisfaction of consumers of the university's educational services are collected, on the basis of which the effectiveness of the network educational program can be evaluated. It seems to us that such studies are too narrow in nature and can only assess one of the aspects of the implementation of network educational programs.

4. Evaluation of the effectiveness of network educational programs is possible on the basis of value added analysis (VA). In this case, an analysis of the activities and business processes of the university is carried out in order to determine the amount of value added as a result of certain operations and processes, as well as the usefulness of services in relation to which consumers. PV can be calculated as the ratio of the value added by the business process for the end user to the value added by the same business process for the end user.

5. The fifth group of methods includes performance evaluation based on a balanced scorecard (BSC). Using the BSC makes it possible to evaluate the effectiveness of a network educational program for various stakeholders: the impact on the financial performance of the organization, customer satisfaction, the impact on the quality of graduate training.

Study existing methods evaluating the effectiveness of educational programs, allows us to conclude that the most preferable approach in evaluating the effectiveness of an educational program is a method based on the BSC, which makes it possible to control the current state and strategic development of this type of educational services.

To select indicators and build a balanced system from them, you can use multi-criteria optimization - the method of analysis of hierarchies (AHI)T. Saaty , which has proven itself in a variety of fields . The MAI makes it possible to form a hierarchy of indicators, determine the scales and criteria for evaluating performance and effectiveness, and determine the significance (weight) of each indicator.

The system of indicators, scales and criteria for assessing the effectiveness and efficiency of the implementation of educational programs in a network form

Based on the results of the analysis of methods for assessing the effectiveness and features of the network form of implementing educational programs, the following hierarchy of enlarged groups of indicators was formed to assess the effectiveness and efficiency (Fig. 1).

Rice. 1. Hierarchy of enlarged groups of indicators for evaluating the effectiveness and efficiency of network educational programs

In this case, when identifying indicators, factors were taken into account that affect the quality of training of higher professional education personnel in terms of:

• training of qualified personnel for priority sectors of the industry and regional economy and the labor market;
• improving the quality of educational programs in priority areas of sectoral, intersectoral and regional development in accordance with international standards;
• development of applied research for the needs of enterprises in the industry and the region.

Performance evaluation

Based on the results of the analysis of the features of network educational programs, the following hierarchy of indicators and indicators was formed. At the top level of the hierarchy, there are only two groups of aggregated target indicators and indicators.

• Achieving the goals of implementing network programs
• Evaluation of the quality of program implementation.

These two groups include indicators and indicators presented in fig. 2.

Rice. 2. Metrics and indicators for performance evaluation

Table 2.

Tab. 2. Hierarchical system of indicators for evaluating the effectiveness of network educational programs

* in the absence of a network program is not implemented, a critical indicator

The objective function of performance evaluation, taking into account the significance of each of the indicators, can be formed as follows.

The effectiveness of the implementation of the work plan to achieve the goals of the program and the required level of quality can be assessed by the formula:

n is the number of targets;

xk - the actual value of the critical indicator "The presence of an agreement on network interaction of partner organizations within the framework of the educational program";

xi is the planned value of the i-th target indicator;

If the performance value is 0, the educational program cannot be implemented.

The maximum value of the objective function is 1.8, which means that all indicators of the network program have been met, and the indicators of the “Quantitative characteristics” group have been overfulfilled.

To make a decision on the effectiveness of the educational program, the mathematical method of dividing the segment in half can be used as a decisive rule. Then, according to the obtained value of the objective function, it is possible to single out the categories of network educational programs presented in Table 3.

Tab. 3. Decisive rules for evaluating the effectiveness of online educational programs

Efficiency mark

Based on the results of the analysis of the concept of efficiency and features of the network form of implementation of educational programs, a hierarchy of indicators was formed, shown in Figure 3.

Rice. 3. Groups of indicators for performance evaluation

In more detail, the scales and significance of assessment indicators obtained with the help of expert assessments and agreed with each other are given in Table. 4.

Tab. 4. Hierarchical system of indicators for evaluating the effectiveness of network educational programs

The objective function of evaluating the effectiveness, taking into account the significance of each of the indicators, can be formed as follows. The effectiveness of the program implementation in the network form is evaluated according to the formula:

n is the number of target indicators;

wi - weight of the i-th target indicator;

xi - planned value of the i-th target indicator;

xacti - actual quantitative assessment of the i-th target indicator.

If the value of the objective function is 0, then the educational program cannot be implemented.

If the value of the objective function is less than 1, then the actual values ​​of the most significant indicators lag behind the plan. In this case, it is necessary to determine the allowable possible backlog range. Expertly, this limit was set as 20%.

If the value of the objective function is greater than or equal to 1, this means that there can be two cases: all indicators of the network program are met and some are overfulfilled, or the overfulfilled value of some indicators compensates for the failure of others.

In the second case, it is necessary to determine the maximum allowable deviation from the planned value of the indicator in case of failure. Expertly, this gap was set at 25%.

To make a decision about the effectiveness of the educational program, you can use a complex decision rule, presented in Table. 5.

Tab. 5. Decisive rules for evaluating the effectiveness of online educational programs

Conclusion

Thus, summing up the above, it can be argued that a balanced scorecard is most preferable as an approach to a comprehensive assessment of such concepts as "effectiveness" and "efficiency" of network educational programs.

On the basis of a balanced scorecard, an evaluation methodology has been developed that includes a hierarchical scorecard, including criteria for an evaluation scale for each indicator, the significance of indicators and target functions for evaluating the effectiveness and efficiency of network educational programs. Two decisive rules are proposed for making a decision on the recognition of a specific educational program as effective and efficient.

The proposed methodology is aimed at ensuring that the joint implementation of network educational programs with large research organizations, medical organizations, cultural organizations, physical culture and sports, etc., leading domestic and foreign universities, has become a common practice for Russian higher educational institutions.

The work was supported by the Federal target program development of education for 2011 - 2015

Reviewers:

Dukhanina LN, Doctor of Pedagogy, Professor, Head of the Department of Pedagogy and Methods of Science Education, National Research Nuclear University MEPhI, Moscow.

Putilov A.V., Doctor of Technical Sciences, Professor, Dean of the Faculty of Management and Economics of High Technologies, National Research Nuclear University MEPhI, Moscow.

Bibliographic link