The expert method assumes and is used for. Expert evaluation: features, methods and results. Modern measurement theory and expert assessments

Expert methods are used in solving prognostic, analytical and design problems associated with non-formalizability and lack of certainty in ideas about organizational and economic objects.

The essence of this method: the experts conduct an intuitive-logical analysis of the problem with a qualitative assessment of the judgment and formal processing of the results.

Peculiarities of the method of expert evaluations: the need for a scientifically based organization of expertise, the use of quantitative methods to evaluate the qualitative judgments of experts.

The expert method can be used in determining the forecasts for the development of objects; when defining goals and objectives, alternative distribution of resources; when making decisions under conditions of uncertainty and risk.

The 1st stage of using this method is the formation of a group of experts. Properties that a specialist needs to include him in an expert group:

- competence (degree of qualification in a certain field of knowledge);

- creativity (the ability to solve creative problems);

– analyticity and breadth of thinking;

- constructiveness (the ability to form specific proposals);

– self-criticism of the expert;

relation to expertise.

For the formation of expert groups, testing, documentation and other methods can be used.

The testing method consists in the fact that on the basis of the developed tests, possible candidates are examined, and a group is formed based on the results of the answers.

The documentary method is the selection of experts according to their objective characteristics, which are contained in their personal documents (work experience, position, academic degree, number of publications, etc.).

Method of appointment - determination by the head of the group of experts from among the employees. The main drawback of the method: the opinion of employees can be consistent, but erroneous, expressing the official position of the organization on this issue (“school effect”). The results of the examination in this case are of interest mainly for internal use only.

The 2nd stage of application of the expert method is the examination.

This stage begins with the choice of a method for interviewing experts. There are individual, group and Delphi methods.

With an individual method, assessments are obtained from each expert through questionnaires or interviews, independent of the opinions of others. Then, after their generalization and processing, the overall, resulting assessment is determined. It is rational to use individual expertise when it is necessary to develop a point forecast of the state of an object, when ranking a set of objects, and in other cases when the most important qualities of an expert are his competence and constructiveness.

The group method provides for obtaining a summary assessment or a general decision from all experts at once through a joint discussion. Its use is expedient when searching for non-traditional solutions, when assessing the characteristics of little-studied objects, i.e., when it is necessary to obtain a creative solution. Group survey can be carried out through discussions, meetings, conferences, brainstorming.

The Delphi method synthesizes a number of positive features of individual and group expertise. Experts independently express their opinion in writing. The most important component of the method is carefully designed survey programs carried out in several rounds, and the regulation of questions at each subsequent round. At the end of each round, the group of organizers of the examination analyzes the answers received, summarizes them and prepares a bulletin based on the results of the round, the text of which is familiar to all experts. At the same time, the information in the help is anonymous. During the second survey, experts receive questions that clarify the initial answers and formulated conclusions, taking into account the results of the previous round. In the third round, the experts are informed on which points there is a common opinion, the experts who expressed a different opinion from the others are asked to substantiate it. The fourth, most often the last, round repeats the procedure of the third. Thus, the area of ​​divergence of opinion narrows and a common solution is developed.

The advantage of the Delphi method is that it reduces or completely eliminates such psychological factors as ostentatious conviction, unwillingness to refuse to publicly express one's opinion, and the influence of authority.

The 3rd stage of expert methods is the processing of the survey results.

To ensure the possibility of formal processing of the results of the examination, a numerical system is needed that describes the properties of objects and the relationship between them using quantitative parameters (various scales of names (classifications), orders, intervals, ratios, differences).

The naming scale is used to describe the belonging of an object to certain classes. Order scale - to measure the ordering of objects according to one or a number of features (rank scale). Interval scale - to display the magnitude of differences between the properties of objects. Relationship scale - to reflect the relationship of the properties of objects, for example, their weight. Scale of differences - if necessary, to determine how much one object is superior to another in one or more features.

The choice of the scale is determined by the tasks of the examination, the characteristics of the object, and the capabilities of the group.

When processing the results of the examination, the choice of the measurement method is important. The most common methods are: ranking, pairwise comparison, direct evaluation, sequential comparison.

The regulation must meet the following requirements: ensure sufficient diversity of wording; unity of the structure of the formulation (for example, the formulation should consistently answer the questions: what is needed? over what (with what)? for what?). The resulting formulations should fully reflect their most important content, i.e., have a significant capacity; wording should be done in such a way as to exclude discrepancies.

The problems of improving expert technologies are related to the development of the following areas: the formation of an expert commission, the organization and conduct of examinations based on the use of modern methods, the use of multi-criteria assessments in interpreting the results.

Expert Methods - evaluation methods carried out by a group of experts under conditions of uncertainty or risk.

Expert Methods are used to determine the nomenclature of quality indicators, their weight coefficients, to measure quality indicators and evaluate them by the organoleptic method. Evaluation of quality indicators by measuring, registration, calculation methods is used to determine the complex quality indicators of various levels of the hierarchy.

Designed for expert evaluation of goods in cases where other previously listed methods are inapplicable or uneconomical.

Expert methods are based on making heuristic decisions, which are based on the knowledge and experience accumulated by experts in a particular field in the past.

Expert methods have certain advantages and disadvantages.

The advantages are that they allow decisions to be made when objective methods are unresponsive. Other advantages include their recoverability. The scope of these methods is not only the assessment of the quality of goods, but also the study of the operations of the technological cycle, decision-making, management, forecasting.

Expert methods, applied by qualified experts, allow for an accurate assessment of goods. The conducted experiments show that with the correct method of expert evaluation, the error of the results is 5-10%, which is allowed for measurement methods. The results of expert assessments obtained experimentally in various groups of experts showed their rather high recoverability.

The disadvantages of expert methods include subjectivity, limited application, high costs for their implementation.

The subjectivism of expert methods is a consequence of the fact that the expert evaluation is carried out by each expert individually and, according to E. P. Raikhman and G. G. Azgaldov, is "nothing but his psychological reaction to the physical and chemical characteristics of the product." However, it should be borne in mind that an expert assessment is made up of the opinions of several experts, each of whom is not only a specialist in a particular field of knowledge, but also a consumer. Therefore, the expert assessment to a certain extent reflects the opinions of consumers, which cannot be done with other methods.

Due to a significant amount of subjectivity, expert methods have certain limitations. their use is rational in two cases: firstly, when the goals set for the experts cannot be solved by other methods; secondly, when available alternative methods give less accurate and reliable results or are associated with high costs.

To eliminate this shortcoming, expert methods are combined with other methods when conducting a commodity examination. Most often, expert and organoleptic methods are used together. Moreover, when developing organoleptic scoring scales, choosing the nomenclature of quality indicators, and determining weight coefficients, expert methods are indispensable.

Expert methods are divided into three subgroups: 1) methods of group survey of experts; 2) mathematical and statistical methods for processing expert assessments; 3) methods for expert assessment of quality indicators.

Each group of expert methods, in turn, is divided into types and varieties. The classification of expert methods is shown in fig. 1.3.

Rice. 1.3. in

Methods of group survey of experts - methods based on conducting a survey of a group of experts with subsequent analysis and processing of information received from them.

The purpose of these methods is to obtain group expert opinion for making final decisions.

The rationale for the choice is the need to make complex decisions in a situation of uncertainty or to make a scientifically based forecast that requires the participation of a group of independent and competent specialists in a narrow field or many fields of knowledge (for example, knowledge of a homogeneous group of goods or all food products).

The main advantages of a group expert assessment are the possibility of a comprehensive analysis of the quantitative and qualitative aspects of the problems of determining and / or forecasting individual characteristics of goods or their combination. Interaction between experts makes it possible to significantly increase the amount of total information owned by a group of experts, in comparison with the information of any member of the group. In addition, the number of factors taken into account in the group assessment and affecting the effectiveness of the decision being made is greater than the sum of the factors taken into account by one expert. With a group assessment, there is less error in making basic decisions and indicators that are not essential for solving the problem. Therefore, an important advantage of group evaluation is the possibility of obtaining a generalized result.

Disadvantages of group estimates include: difficulties in obtaining a reliable and consistent estimate; receiving unequal answers to the same question with a large difference of opinion due to the different competence of experts; receipt of unambiguous answers does not guarantee their validity and reliability, and this cannot be verified during the examination; more incorrect information from a group of experts than from an individual expert can lead to significant errors in the final results; the possibility of confrontation, when individual experts for uncertainty or other reasons may agree with the opinion of the majority.

Despite these shortcomings, it has been experimentally established that, subject to certain requirements, a group assessment is more reliable than an individual one. These requirements include: an acceptable distribution of grades; group reliability; examination preparation.

The effectiveness of the examination depends on the accuracy and reliability of the results obtained, that is, on the methods used and on the qualifications of the expert. The choice of an expert is a difficult task, most often several personal characteristics are taken into account: competence - professional and qualimetric, interest of the expert in the results of the examination, attitude to the case, objectivity. Almost no account is taken of such traits as risk appetite and other psychological characteristics.

A systematic approach to assessing the quality of an expert has not yet been developed; existing assessment methods are divided into five groups:

heuristic;

Statistical - estimates that are obtained as a result of the analysis of expert estimates with the determination of deviations from the average values;

Test - assessments obtained as a result of the performance of test tasks by experts;

Documentary - assessments of competence obtained in the analysis of individual documentary data of examinations conducted by an expert;

Combined - estimates obtained during the analysis of data obtained by combining the above methods.

Heuristic assessments include self-assessment and assessment made by a team of experts. Self-esteem is most often subjective; to reduce subjectivity, a point scale is used for individual properties of an expert. Self-assessment is carried out according to the types of goods and quality indicators, for example, the aesthetics of products, the expert evaluates himself by questioning. The questionnaire includes the frequency of familiarization with modern domestic and foreign literature, with modern product samples, with the results of sociological surveys.

As a kind of self-assessment, the method of assessment by reasoning and familiarization with the products that are being analyzed is used. The assessment is carried out using a questionnaire with the determination of the coefficient of competence.

By using test grades it is possible to evaluate such important properties of an expert as qualimetric and professional competence, objectivity. Qualimetric competence in carrying out an organoleptic assessment consists in a multiple survey of experts to determine the weight coefficients several times and determine the reliability of the assessments.

Professional competence is tested in the form of a control, the ability to use various types of rating scales - order, relationships, intervals, as well as the ability to distinguish a significant number of characteristics, gradations when evaluating the properties of the product being analyzed.

With the participation of experts in a collective assessment and discussion, the phenomenon of conformism is observed, that is, the expert falls under the influence of the conclusions of other experts, which negatively affects the formation of an objective assessment of the expert. Therefore, the ability to adhere to one's opinion and defend one's conclusions is of positive importance during the examination by a collective method and the formation of an objective assessment.

Expert Methods quality assessments are based on the use of expert opinions. They are used when it is impossible or inexpedient, uneconomical to use measuring or calculation methods. This happens when there is a lack of information, the need to use and develop special technical means, when evaluating aesthetic quality indicators, and the like. Expert methods can be combined with other methods or used as an independent type when assessing the quality of regulatory documentation for products and products, determining the range of indicators and their weight coefficients, when choosing basic samples and quality indicators, when determining and measuring quality indicators by the organoleptic method, during the time for evaluating single and complex quality indicators determined by the measuring or calculation method.

The basis of expert evaluation of quality is the objective social utility of products, which reflects its modernity. The opinion about the quality of products expressed by a qualified expert meets public needs and coincides with the opinion of the mass consumer.

However, in choosing a product, the consumer adheres to a conservative position and accepts new products carefully, and sometimes does not perceive them at all. In the opinion of experts on the quality of the goods, a set of opinions of consumers of products is summarized. Studies have shown that the quality assessments of the experts' goods coincide with the assessments of product quality that were obtained during a mass survey of consumers. Determination of the accuracy of expert studies for compliance with the methodology of the examination showed that it is 5-10%.

In relation to product quality, its consumer properties, expert methods are used in such cases:

Determining the nomenclature of indicators during the examination;

Selection of criteria for evaluating a product or product and placing indicators according to the principle of hierarchy;

Determination of weight coefficients of quality indicators to determine the quality level of a product, goods;

The study of indicators by the organoleptic method using the methods of their quantitative expression;

Evaluation of quality indicators by measuring, registration, calculation methods to determine the complex quality indicators of various levels of the hierarchy.

The most commonly used expert methods are:

Leading expert (single);

commissions;

combined.

The leading expert method makes it possible to quickly conduct an examination, reduce the time for the coordination and discussion procedure, and statistical processing of data by group members. However, the results of an examination conducted by one expert depend on the level of his professional knowledge, personality, and level of competence.

The method of expert commissions involves the participation of a group of specialists who conduct analysis and evaluation. This method allows you to obtain reliable, objective results, but requires a significant investment of time in the preparation and organization of the examination. The number of experts who form the commission depends on the required accuracy and reliability of the results of the examination. The expert commission consists of two groups - working and expert. The working group prepares, organizes and conducts an expert assessment of product quality, followed by analysis of its results. The working group includes an organizer, a consultant who has professional knowledge about the products, and technical workers evaluate them. The expert group may consist of several subgroups, each of which specializes in solving the relevant tasks - determining the nomenclature of indicators, evaluating individual groups of indicators, and the like.

The combined method, which is based on the consistent use of the work of a leading expert and a small number of expert commissions, is used in some cases.

There are no expert methods in the General Classification. However, classification is used depending on the ratio of the amount of data obtained by the expert or analytical method, the method of obtaining information from the expert, and some other factors.

Depending on the method of obtaining information from an expert, methods are distinguished:

Collective;

Individual.

With a collective method, a technical worker conducts a survey of the entire group of experts at once, with an individual method, each expert separately. To obtain reliable results, it is necessary to clearly and correctly set the goal and objectives for the expert, with the group method it is difficult, but possible with a constantly formed group. With an individual method, interviews, interview questionnaires, questionnaires, mixed questionnaires are used. Surveys can be conducted face-to-face and by correspondence. In the face-to-face method, the expert expresses his judgments to the person conducting the survey. During a remote survey, there is no contact between them and the expert fills out the questionnaire or survey card himself.

During the interview, the technical worker makes a note in the form of a conversation that goes through the program and a specific list of questions. During the interview questionnaire, the list of questions is more specific, directed, the sequence of questions is strictly defined. The questionnaire is filled out in the presence of an expert.

Questioning is different in that the expert independently fills out the questionnaire, having an explanatory note on filling it out. Mixed questioning involves a preliminary explanation of the expert on filling out the questionnaire with a specification of the task.

In terms of informativeness, interview methods, interviews - questionnaires, and questionnaires have the greatest opportunities. The greatest independence of judgments is characteristic of the questionnaire method.

To solve complex situations of uncertainty or during the formation of a scientific and technical forecast, expertise requires the participation of a group of erudite specialists who are well-versed in many fields of knowledge. The main advantage of the collective assessment lies in the possibility of a comprehensive analysis of the quantitative and qualitative aspects of the problems. There are problems that cannot be solved without the participation of specialists. It is assumed that the opinion of a group of experts is more reliable than the opinion of an individual, that is, two groups of equally competent experts are more likely to find an objective solution.

The combination of operations for preparing and conducting surveys of experts, as well as technical operations, made it possible to create several expert methods that have received the greatest recognition and distribution. These include the Delphi, PATTERN and combined methods.

Delphi method (in some sources - Delphi) - a method of interviewing experts based on sequentially implemented procedures that are aimed at forming a group opinion on procedures with insufficient information.

The Delphi method was developed in the American research company RAND Corporation by W. Helmer, N. Dolki and T. Gordon. It was used for military scientific and technical forecasting of the future. The term Delphi comes from the name of the town in Greece, where the oracles lived at the temple of the god Apollo.

The features of the Delphi method are: the refusal of the joint work of experts; anonymity of estimates; adjustable feedback; group response.

Refusal of joint work of experts and anonymity are achieved by the fact that each expert expresses his opinion in the questionnaire, without group discussion. Other techniques of individual survey are also used, for example, answers to questions are entered by experts into a computer. This allows you to reduce the discrepancy between individual estimates and get a group answer, which correctly reflects the opinion of each expert.

The anonymity of the survey allows to reduce the conformity of the authoritarian influence of individual dominant experts, regulated feedback reduces the influence of individual and group interests. The introduction of feedback also increases the criterion of objectivity and reliability of estimates.

When using this method for the purposes of expert assessment of the quality of consumer goods, the following disadvantages appear: the complexity of interviewing experts and filling out questionnaires, the complexity of the assessment due to the large number of quality indicators (sometimes up to 20-40) and filling out several questionnaires (3-10), cumbersome records of explanations due to the lack of direct contact between the organizer and the experts.

The method is promising for obtaining a group expert assessment and in-depth analysis of events in situations of uncertainty.

PATTERN Method - a method of interviewing experts based on building a hierarchical structure - a tree of goals - and making a decision on these goals after an open discussion.

The name of the method consists of the first letters of English words, meaning "Assisting planning by quantifying technical data."

The method was developed by the American company "Honquell" for evaluating projects of new weapons systems. The method has analogues: PROFILE, in France - the KPI method, PROPLEN etc.

The PATTERN method involves several stages.

Stage I - the formulation of the main problem that needs to be solved, and its division into a number of secondary problems of the first, second, etc. order, which are then divided into narrower tasks. The division continues until simple elements are obtained that can be evaluated by experts.

As a result of this division, a hierarchical structure of main, secondary problems and tasks connected with each other is obtained, called the goal tree.

Stage II - determination with the help of experts of the coefficients of weight (or significance) of each task in relation to the main goal, while the experts make a decision after an open discussion in the expert group.

Such an open discussion, along with a positive factor - the interaction of experts who strive to make a positive decision - also has negative consequences due to conformism, i.e., distortion of the real opinion of experts due to suggestion or adaptation to the opinion of the majority.

Stage III - the use of computers for processing the received data and their analysis. The advantage of the PATTERN method is the simplification of the expert survey procedure. Disadvantages: lack of rationale for the optimal number of members of the expert group, as well as the methodology for selecting competent specialists for the expert group; processing the results of the survey without taking into account differences as individual experts; lack of barriers to the manifestation of expert conformism; insufficient development and uncertainty of the principles of building a tree of goals.

Since the Delphi and PATTERN methods have significant shortcomings and do not fully correspond to the goals of peer review, Oe. L. Raikhman and G. G. Azgaldov proposed a combined method in which the positive features of other expert methods were used and their shortcomings were excluded.

Combined method - a method based on a combination of individual and collective expert assessments.

The advantages of the combined method are sufficient flexibility, which allows eliminating errors in the survey of experts and increasing the reliability of the examination results, a clear definition of the strategy by classifying tasks according to the degree of significance and operations for their implementation, high reproducibility of the results.

The disadvantages of the method include a rich operation, which requires a significant investment of time and money. However, this shortcoming is compensated by increased reliability and recovery of results.

To assess the quality of goods, the combined method has a general algorithm of expert operations:

1. Preparatory stage:

Formation of a working group;

Formation of an expert group;

Classification of products and consumers;

Building a block diagram of quality indicators.

2. The stage of obtaining individual expert assessments:

Selection of the procedure for assigning assessments by experts;

Choosing a method for obtaining information from an expert and preparing documents required for the survey;

Survey of experts.

3. The stage of obtaining collective expert assessments:

Generalizations of individual expert assessments;

Determining the consistency of individual expert assessments;

Determination of the objectivity of collective expert assessments.

There are corresponding tasks for each stage. During the preparatory stage, tasks such as determining the functions and structure of the working group, its quantitative composition, and the responsibilities of individual members are solved. The principles for the formation of an expert group are determined and developed: regarding the number of experts, their professional training, and the like.

At the second stage, the technique of interviewing experts is determined, the presence of contact between experts, the method of transmitting information and the form of expert assessments are evaluated. Experts can determine the scores on their own or after discussion with other experts or familiarization with anonymous experts. The scores are justified, quantified and dichotomous, where answers are given in the form of "yes", "no" or 0-1.

Rational use of information received from experts is possible if it is converted into a form convenient for analysis, preparation and decision-making. The possibilities of converting information into appropriate forms depend on the specific features of the object, the completeness of data about it, reliability, the level of decision making, as well as on the accepted criterion, depending on the problem under study.

One of the elements common to many expert methods is the weight coefficient.

The weighting coefficient is a quantitative characteristic of the degree of significance of a particular indicator for assessing quality.

The determination of the weighting coefficients of quality indicators is carried out by the expert method. The weight coefficients are intended to increase the reliability of the expert assessment of the quality of goods.

Each indicator occupies a certain place in the nomenclature of quality indicators in terms of importance. Experts rank indicators according to the degree of significance based on professional knowledge and skills. In addition, any qualified expert seeks to evaluate the quality indicators of the examined goods from the standpoint of the mass consumer.

If an expert has the ability to compare and evaluate possible options for action, giving each of them a certain number, then he has a certain system or scale of preferences. The correct application of the scales is essential to ensure the accuracy of expert judgments. There are the following types of scales: nominal, ordinal, interval, relations. But the order scale has the greatest advantages over others due to the relative simplicity of expert evaluation of quality indicators in terms of significance. The characteristics of the scales are given in Table. 1.1.

Nominal name scale used to distinguish one object from another. Objects should be numbered, however, the numbers indicate the object, and not its quantitative characteristics. This is a simpler type of measurement that only uses numbers or symbols to classify objects. The scale can be used for digital coding of individual properties in questionnaires, to determine weight coefficients.

Ordinal scale (ranks) - such an evaluation method in which the parameters, indicators or objects that are evaluated are arranged in the order of increase or decrease in the indicator of the parameter (indicator) or the properties of the object. A classic example of evaluation using an ordinal scale is the assessment of the hardness of minerals on the Mops scale (the scale of relative hardness consists of 10 hardness standards, with the hardness of talc taken as 1, diamond - 10). This method can be used to determine the intensity of the color of flour, the aroma of fruit juices, the bouquet of wines, the texture of cheeses. The ordinal scale has advantages when used to determine weighting factors, since it simplifies the process of ordering quality indicators in terms of importance for consumers.

Table 1.1. Scale types and their characteristics

Scale type	Scale Definition	Relationships set on the scale
Rated	A simple measurement type that uses numbers or symbols only to classify objects.	Equivalence (=)
Ordinal (ranks)	Objects of one class are in a corresponding relationship with 3 objects of another class (more than, more advantages, stronger, etc.). If [A]>[B] for some objects of classes A and B, then there is a partially ordered scale.	Equivalence (=). Greater than (>).
Interval	The ordinal scale, which is divided into known distances between any two numbers on the scale, the zero point of the scale and the unit of measurement are chosen arbitrarily.	Equivalence (=) Greater Than (>). The ratio of any two intervals is known.
Relations	Interval scale using a true zero point, the ratio of any two points is independent of the unit.	Equivalence (=). Greater than (>). The ratio of any two intervals is defined. The relationship between any two points is defined.

When determining the weight coefficients of quality indicators, experts first evaluate the most important of these indicators (in their opinion) and assign it a certain number, for example 1. All further indicators are evaluated in decreasing or increasing order of importance.

After that, the data of all experts are averaged for each indicator.

In the practice of assessing the quality of goods, a method for determining weight coefficients, called the "fixed sum method", is often used. Its essence lies in the fact that experts assign a weighting coefficient for the indicators included in the upper level indicator, and the sum of these coefficients must be equal to a predetermined number.

The "fixed sum method" is expediently used only for a small number of indicators.

Practical experience shows that it is advisable to apply the following procedure for determining the weight coefficients in the expert assessment of the quality of goods.

1. Preliminary ranking by experts of indicators of a homogeneous group. Rank 1 is assigned to the most important indicator, 2 - to the next most important, and so on. If the indicators are equivalent in importance, then they are assigned the same ranks. The number of indicators in a homogeneous group should be 4 or more. If the number is less, the ranking is not carried out.

2. Determination by experts of coefficients of weighting of indicators. An indicator of the 1st rank is assigned a weighting factor of 10. The weighting factor of the next most important indicator is determined as the proportion of the importance of the first indicator. When determining the third and subsequent indicators, their importance in comparison with the previous ones is taken into account. As a result of these successive actions, the expert determines the weight coefficients of single, and then complex quality indicators.

3. Familiarization of experts with the values ​​of weight coefficients (their justifications) assigned by other experts.

Justification of weight coefficients is a very laborious operation, therefore it is used with a limited number of indicators (about 10-15). Otherwise, the experts are invited to give justification only for some indicators at their discretion.

4. Averaging the values ​​of weight coefficients determined by all experts. Conducted by technical workers by calculating the arithmetic mean or weighted average. In the latter case, a comprehensive assessment of the expert's quality is taken into account.

The disadvantages of the scale are the inaccuracy of ranking estimates due to the absence of an equation for intervals, the impossibility of calculating even the arithmetic mean value.

Interval scale. This is such an estimation method, in which the essential characteristic is the difference between the values ​​of the estimated parameters, which can be expressed by the number of units provided for this scale. With the help of such a scale, objects are ranked, and it is also established in certain units how much one object is larger than the other. An example of an interval scale is the Celsius scale, which is divided into 100 equal intervals and is used to characterize such product properties that are associated with temperature conditions, for example, the frost resistance of synthetic leather, the minimum temperature of the freezer in the refrigerator.

Relationship scale allows you to achieve the highest level of measurement. This is an estimation method that uses a unit of measure, it is used for most parameters that are physical quantities: size, weight, density, force, voltage, frequency, and the like. Scale measurement results have the properties of numbers that can be subjected to statistical processing. An example of such a scale is the Kelvin temperature scale, which starts at absolute zero.

When comparing the scale of intervals and ratios, it was determined that with the help of the latter, more accurate results are obtained. In addition, scores obtained from the interval scale can be used to calculate weighted averages, the calculations of which are typical for experimental surveys. The ratio scale can be the basis for the expert method, the order scale can be used with sufficient justification.

In the practice of peer review, two main types of scales are used - dimensional and dimensionless. The data of dimensionless scales are expressed in fractions of a unit, percent, points.

Mathematical and statistical methods for processing expert assessments are methods designed to increase the reliability of the results of assessing the quality of goods by experts.

They are divided into four subgroups: ranking, direct evaluation, sequential preferences and pairwise comparisons.

Ranging - a method based on the arrangement of objects of expertise in ascending or descending order.

Designed to solve many practical problems, when the objects that determine the final results are not amenable to direct measurement. In addition, individual objects, characterized by different nature, are incompatible, because they do not have a common measure of comparison. The basis for ranking is the need to order any object in time and space, as well as in accordance with the measurable quality without making precise measurements. And finally, in a situation where the quality that is being measured cannot, in principle, be measured for practical and theoretical reasons.

The ranking procedure consists in placing objects by an expert in the most rational order and assigning them a certain rank in the form of a natural number. In this case, the most important object gets rank 1, and the least important one gets rank n. As a result, an order scale was obtained in which the number of ranks is equal to the number of objects. If two objects have the same ranks, then they are assigned the so-called standardized ranks, which are calculated as the average of the sum of the places of objects with the same ranks.

For example, six objects are assigned the following ranks:

Objects 2 and 5 shared 2nd and 3rd places. their standardized rank will be

(2 + 3) /2 = 2,5.

Items 3, 4, and 6 are tied for 4th, 5th, and 6th places, and their standardized rank is 5:

(4 4-5 + b) / 3 = 5. The result is the following ranking:

The ranking method is rarely used in its pure form. Most often, it is combined with the direct assessment method or its modifications (ranking by the sum of assessments, a combined method, etc.).

Method of direct evaluation lies in the fact that the range of change of any quantitative variable is divided into several intervals, each of which is assigned a certain score in points, for example, from 0 to 10. The rating scale can be positive and negative, for example, from +3 to -3.Expert should include each object in a certain interval depending on its value. The number of intervals into which the entire range of quality change is divided may be different for different experts. Individual experts are allowed to evaluate qualitatively different factors with the same number.

In some cases, it is more convenient to select the most preferable factor first to evaluate and then to rank.

The total ratings of the ranks can be normalized, this allows you to establish a closer relationship between the ratings that the experts provided to individual objects. For this purpose, the estimates for all objects are summed up and then each of them is divided by the amount received. The normalized scores calculated in this way can be ranked again.

When conducting an examination by several experts, they seek to obtain an average estimate for each object. For this purpose, the normalized estimates of each object are summed up, the resulting sum is divided by the number of experts. The second way to determine the relationship between the ratings of factors is that the most important factor is given an estimate (weight is set) for a certain number of 1 or 10, the following factors are evaluated as a share of the most important factor. The advantage of the method is that it simplifies the process of selecting estimates, because it is not necessary to compare the entire series each time, but only take into account the value of the first and previous estimates in importance. Estimates are averaged by calculating the arithmetic mean.

Successive advantage method - based on comparing an individual object with the sum of subsequent objects to establish its importance. used in measuring the level of quality, evaluating the activities of scientific organizations. The method has such a major advantage over other methods that it makes it possible to compare and measure qualitatively different factors.

The method was developed by V. Churchman and G. Akof and is intended for comparisons within certain tolerances.

The order in which results are presented or grouped does not affect the benefits.

The procedure for successive comparisons is as follows. The expert is presented with a number of objects (indicators, factors, results) that need to be assessed by their relative importance (significance), and he performs the ranking. The most important object is assigned a score equal to 1, the remaining objects - scores below 1 to 0 in order of their relative importance. Then the expert determines whether the object with a score of 1 is more important than the sum of the last factors. If the importance of an object is high, then it increases the score so that it is greater than the sum of all the others.

If the value of an object is lower than the sum of all others, it adjusts the estimates.

Thus, the procedure used is to systematically check the scores by comparing them sequentially.

The method of successive preferences is advisable to apply if the number of compared objects does not exceed 7. With a larger number of objects, they must be divided into subgroups that include 6 objects. Where this is not possible, paired comparisons should be used.

Pairwise Comparison Method - based on comparing the objects of expertise in pairs to establish the most important in each pair.

It is used to identify advantages among a significant number of factors, problems, indicators. Experts can simply make a comparison with a statement of the superiority of one factor over another. It is possible to use a special preference scale, where each degree of advantage has its own specific assessment.

The method of paired comparisons can also be used to establish the total ranks of factors.

To facilitate the procedure, matrices of pair equations are compiled in which all objects (factors) are written in the same order twice: in the top row and the leftmost column. Each expert must put down at the intersection of the row and column an ​​estimate for the two compared factors. Depending on which factor is the most important, this score will be equal to 1 or 0, respectively. Dashes or zeros are put in the main diagonal of such a matrix (table. 1.2).

Table 1.2.

Each pair of factors can be compared once or twice. There are various options for partial pairwise comparison: choosing the preferred object from pre-grouped pairs; partial pair comparison of one group of objects with all others, while the remaining factors are compared with some others; establishment of total ranks of factors.

Paired comparisons are sometimes combined with a preliminary ranking of objects, whereby paired comparisons are used to clarify the advantage of individual objects. In this case, an additional matrix is ​​constructed, which indicates the proportion of cases where one factor is more significant than the other, in the total number of estimates obtained.

Methods for expert evaluation of product quality indicators- these are methods for determining the actual values ​​of single and complex quality indicators.

Designed to determine the values ​​of the quality index by calculation or heuristics in cases where the use of measurement methods is impossible or uneconomical due to excessive costs for their application or a long test time. For example, when determining the taste and smell of food products, only organoleptic methods are used. Measuring methods do not give an accurate reliable estimate, despite the increased costs.

For differential and complex evaluation of samples that differ significantly in quality, it is recommended to determine the value of a single indicator P as follows:

where P;5 is the base (reference value).

Another, more accurate method is based on the study of indicators in order to determine the types of dependence and, i.e., in order to develop formulas for calculating indicator estimates:

Evaluation of single quality indicators begins with the determination of the permissible intervals for their change (P; - P;) P; - the best value of the indicator, the excess of which is impractical or impossible. The principles for assigning the maximum allowable value of an indicator depend on the purpose of quality assessment, while it is necessary that this principle be the same for all indicators.

Increasing the reliability of expert assessments is achieved by dividing complex operations into simple ones, which make up a multi-stage procedure for assessing the permissible values ​​of the indicator. The transition to each next level is carried out after the adoption of agreed decisions on the previous one.

The expert procedure for determining the acceptable values ​​of quality indicators consists of a number of operations:

issuance of questionnaires and explanatory notes to the experts, which list the quality indicators and describe the principles for choosing the acceptable values ​​of the indicators;

filling out questionnaires by experts and indicating specific models of products, the values ​​of which they consider to be the maximum allowable;

familiarization of each expert with the assessments indicated by other experts, and their discussion;

conducting the second (sometimes third and fourth) round of the survey;

Averaging the evaluation results.

With a significant difference of opinion, an additional round of voting is held. The value of the indicator is taken as the maximum if at least 70% of the votes are cast for it. If this condition is not met, the average of 50% of the largest values ​​of ріmax is taken as the maximum allowable value, the average of 50% of the largest values ​​is taken as the minimum allowable value, the values ​​are used by experts when determining the estimates of quality indicators K.

To determine by experts the type of dependencies (i) between the value of indicators G; and their estimates of K, the "principal method of points" is often used. The need for its use is due to the fact that the division of the estimation procedure into several stages simplifies the work of the expert and allows him to give estimates to some characteristic points, on the basis of which it is possible to build a model of the real value.

"Principal point method" There are several varieties depending on their number.

"Method of the three main points" - based on the separation of the values ​​of indicators G; on the maximum, minimum, average values ​​and determining the values ​​of the estimates P; at these points. The scale interval between the maximum and minimum points is preset (0-1 or 0-10 scales). The expert's task also includes determining the dependence trend in the interval between the main points and plotting. After that, from the graphical dependence, you can go to the analytical formula for calculating the estimates of the quality indicator Ki "Method of the three main points" allows you to develop only an approximate evaluation model.

"Method of the seven main points" - a method for evaluating indicators on a seven-point scale for evaluating indicators, the values ​​of which are determined experimentally or by calculation, as well as by the organoleptic method.

The seven-point scale is uniform, that is, when moving from one quality class to another, the score changes by one point. These scales have found wide application, especially in organoleptic evaluation. To obtain more accurate results, one should proceed to determining the type of relationship between grades and points.

To facilitate the work of the expert, five graphs are given in the explanatory note to the questionnaire. The expert selects the curve (or combination of curves) that, in his opinion, best reflects the nature of the dependence and Then, each quality class is assigned a score in accordance with the nature of the dependence and the values ​​of the quality indicators. In this case, it is advisable to use numbers in the range 0-10, multiples of 0.5, and the "highest quality" class receives a score of 10.

Thus, the graph constructed by the expert characterizes the relationship between the absolute values ​​of Pi indicators and their estimates K;, and for indicators evaluated by organoleptic methods, between quality classes and their estimates.

In conclusion, the obtained results are discussed, processed and analyzed. For indicators that are determined by measuring and calculation methods, it is desirable to give an analytical description of the curves, which makes it possible to calculate an estimate for any values ​​of the indicators.

The use of the "principal points method" makes it possible to group and classify indicators according to the types of dependence.

The definition of complex quality indicators is carried out by two types of methods:

methods of comprehensive assessment of the quality of product samples;

methods of building models of complex quality indicators.

There are two varieties of integrated quality assessment methods - the express method and the methods of moving through levels without training and with training.

Express methods for comprehensive assessment of the quality of product samples are based on the determination of a complex quality indicator by analyzing the values ​​of individual single indicators and appearance without their preliminary assessment and taking into account the weighting coefficients.

When using these methods, it should be taken into account that the maximum number of estimated indicators, even for a highly qualified expert, is 7-9 indicators located at the same level of the hierarchy, which make up a fairly homogeneous group. In addition, experts should take into account the importance of individual indicators with the help of weight coefficients, the relationship between them, and also consider the quality of the product as a system.

The method of moving through the levels without preparation is a set of operations that are carried out sequentially, with a gradual increase in the level. In this case, the analysis starts from the lower level of the indicator tree. Taking into account the value of the indicators of the lower level, the expert evaluates the indicators of the higher level. These operations are repeated with increasing level until the upper level is reached - a comprehensive (generalized) quality assessment.

Method of moving through levels with preparation based on a preliminary determination by experts of the weighting coefficients of quality indicators and their assessments. When assigning complex assessments, the expert knows the average values ​​of the weighting coefficients and estimates of individual indicators. The procedure for determining complex indicators is similar to the procedure for the method of moving through levels without preparation.

Formalization of the peer review process is to find the relationship between the values ​​of quality indicators G; (or their estimates K) and the quality indicator of the higher level, i.e., in determining the type of decisive function that experts use when assigning complex indicators. At the same time, the decisive function, like any model, simplifies the object of study, since not all indicators and the relationships between them are taken into account.

The initial data for determining the type of function can be the results of evaluating the quality of various samples by express methods or methods of moving through levels. Then the estimates assigned by the experts are reduced to a common matrix, each row of which is a set of estimates of individual indicators of the sample and complex expert estimates. Based on this, machine algorithms can be developed and programs can be compiled to find the decisive functions with the help of a computer.

The expert method for determining quality indicators is based on taking into account the opinions of expert experts. An expert is a specialist who is competent in solving a specific problem (from the Latin word “expertus” - experienced). This method is used in cases where certain quality indicators cannot be determined by other methods due to insufficient information, the need to develop special technical tools, etc.

The expert method is a combination of several different methods, which are its modifications. Known varieties of the expert method are used wherever the decision is based on a collective decision of competent people (experts). So, for example, the decisions of various councils, conferences, meetings, commissions, as well as examiners in assessing students' knowledge, etc. - all these are decisions made by expert methods.

Classification features that allow grouping expert methods are: taking into account the competence of experts in the formation of an expert group; methods of interviewing experts; ways of exchanging information; types of scales used for evaluation. For expert evaluation, it is advisable to use scales with an odd number of gradations, in which, in addition to the lower and upper levels, there is an average (satisfactory) level.

Expert methods can be used to immediately form a general assessment (without detailing) of the level of product quality, as well as in solving many particular issues related to the definition of quality indicators. The areas of application of expert methods are as follows:

• assessment of regulatory documents for products;
• generalized assessment of product quality;
• classification of goods being valued;
• determination of the nomenclature of quality indicators of the evaluated products;
• determination of weight coefficients of quality indicators;
• selection of basic samples and determination of the values ​​of basic quality indicators;
• measurement and evaluation of quality indicators using the senses (organoleptically);
• assessment of single indicators, the values ​​of which are determined by measuring or calculation methods;
• product identification;
• determination of a complex quality indicator based on a set of single (generalized and group indicators);
• assessment of product competitiveness;
• product certification.

The expert method is not used if the quality can be assessed by other methods with greater accuracy or at a lower cost.

The results of an expert assessment have elements of uncertainty and groundlessness. The reliability of the assessment results depends on the competence and qualifications of the experts.

The competence of an expert consists of professional and qualimetric competence. Professional competence provides knowledge in the field of:

• the history of the development of the evaluated products, changes in its properties and quality indicators;
• conditions for the design (construction) and production (processing) of products;
• values ​​of quality indicators of domestic and foreign analogues;
• prospects for the development of products, reflected in research works, patents, design developments;
• consumer requirements, conditions and nature of consumption (operation) and disposal.

Qualimetric competence provides a clear understanding of: an approach to assessing the quality of goods; quality assessment methods; issues of construction and application of rating scales; determination of subjective probabilities and differences in a sufficient number of gradations of the evaluated object.

The qualification of an expert is determined not only by knowledge of the subject of discussion, but by the ability to express clear, unambiguous judgments. In addition, the specific capabilities of the expert are taken into account. For example, in the food industry, when assessing the quality of food products, the ability of an expert to perceive taste, smell, etc., as well as his state of health, is taken into account. Assessors evaluating aesthetic and ergonomic performance should be well versed in the field of artistic design.

When using the expert method for quality assessment, two groups are formed: working and expert. The working group organizes the procedure for interviewing experts, collects questionnaires, processes and analyzes expert assessments.

The expert group is formed from highly qualified specialists in the field of creation and use of the evaluated products. These can be merchandisers, marketers, designers, designers, technologists, etc. The number of experts in the group depends on the required accuracy of average estimates, the allowable complexity of the assessment procedures, the ability to manage the group and the capabilities of the organization in which the group is formed, but the group must be at least 7-12 people. If it is necessary to improve the accuracy of quality assessments, the composition of the group can be increased to 15-20 experts.

It is desirable that the expert group be formed not for a one-time examination, but as a permanently functioning body with a fairly stable composition of experts. During the work of the group, the experts, based on the analysis of the results of previous work, develop common approaches and principles for assessing product quality, which increases the effectiveness of the assessment.

When assessing quality, experts are invited to build a hierarchical nomenclature of quality indicators. When constructing a hierarchical nomenclature of quality indicators, it is desirable to go down to the level of consideration at which there are single indicators for which there are objective evaluation methods.

When constructing the nomenclature of quality indicators, it is advisable to fulfill the following four conditions:

• 1) a sign by which any complex indicator is divided into P indicators of the lower level (classification feature), should be the same for all? indicators. This provides experts with the best opportunities for comparing indicators when determining weight coefficients (the next operation after building the nomenclature);
• 2) determining the weighting coefficients, the expert compares the importance of various quality indicators included in a homogeneous group. Since this operation becomes difficult with a significant number of indicators, which reduces the objectivity of the results, the number of indicators included in a homogeneous group should not exceed 10;
• 3) if the quality indicator is repeated at two or more levels, then its weight coefficient is considered too high. Therefore, the repetition of indicators is undesirable;
• 4) the number of quality indicators included in homogeneous groups at the same level of the nomenclature of quality indicators should not differ sharply, since an increase in the number of indicators can lead to a decrease in the values ​​of weight coefficients. For example, in the nomenclature of quality indicators at the 1st level there are three indicators (5. = 1, 2, 3), at the 2nd level each of them consists of a certain number of indicators - respectively q, I, p. It is desirable that the condition q ~ I~ p or so that these numbers are close enough.

Possible errors can be reduced by including the necessary explanations in the explanatory note.

The work of experts consists in performing two independent, but interrelated operations - correcting (clarifying) the nomenclature of quality indicators and determining the weight coefficients of these indicators.

After studying the presented nomenclature of quality indicators, each expert comes to one of the following conclusions:

• a) in the nomenclature there are quality indicators, the weight of which is insignificant. They can be ignored when assessing quality (the expert crosses out these quality indicators);
• b) the nomenclature does not include sufficiently important indicators (the expert enters these indicators into the nomenclature);
• c) crosses out indicators that he considers unimportant and at the same time supplements the nomenclature with indicators that he considers sufficiently important;
• d) the nomenclature of quality indicators is compiled correctly.

The expert sets out the motives for his actions on a separate sheet attached to the nomenclature, or on a technical worker who fills out this sheet.

After analyzing the opinions of experts by members of the working group, the nomenclature is corrected and again submitted to the experts to determine the weighting coefficients of quality indicators.

Determination of weight coefficients of quality indicators

The determination by experts of the weighting coefficients of quality indicators begins with ranking, when each quality indicator is assigned a certain rank. If a homogeneous group includes four or more indicators, the experts preliminarily rank them: rank 1 is assigned to the most important indicator, 2 - to the next in importance, etc. If the expert believes that the weight of two or more indicators is the same, then he assigns them the same ranks . If there are less than four indicators, the ranking operation can be omitted.

Each expert gets acquainted with the values ​​of the weighting factors assigned by other experts and their justifications. If the expert group is formed from employees of various organizations who are difficult to gather for joint work, then each expert attaches a brief justification to the completed questionnaire. Since this procedure is very time-consuming, it is recommended to use it when the number of indicators included in the block diagram is relatively small (about 10-15). Otherwise, experts are invited to justify only some values ​​of the weighting coefficients at their own discretion. To simplify the procedure, you can opt out of justifications and request them only if necessary.

Each expert gets acquainted with the anonymous opinions of other experts and again puts down the values ​​of the weighting coefficients.

If it is not difficult to gather an expert group for joint work, then an open discussion of all weighting factors is held. All experts have the opportunity to briefly argue their judgments about the value of the weighting coefficient of each indicator and criticize other opinions. In order to exclude the possible influence of official position on the opinions of experts, it is desirable that experts speak in sequence from junior to senior (according to their official position). After the discussion, the experts put down the values ​​of the weighting factor of the quality indicator and move on to the next indicator.

According to the results of the evaluation of the weighting coefficients, it is estimated consensus of expert opinions using coefficients of variation, concordance, etc.

The values ​​of the coefficients of variation are determined by a number of factors: the number of levels in the nomenclature of quality indicators, the diversity of consumer opinions, the competence of experts, etc. When determining the weighting coefficients, the following values ​​of the coefficients of variation, obtained based on the analysis of the results of the work of various expert groups, can be recommended: V 0.35 - Consistency is low.

If the consensus of experts' opinions is average or above average, then it is possible to proceed to the assessment of the consensus of opinions of the group of experts on the weight of all indicators. Below average agreement requires additional analysis. The reasons for the low consistency of expert opinions can be subjective and objective.

Subjective - insufficient awareness of experts about the quality indicator, the weight coefficient of which is determined; fuzzy understanding of the problem being solved; arithmetic error of experts, etc.

Objective reasons - the conducted classification of consumers, on which experts rely when determining weight coefficients, is insufficient. Therefore, it is necessary to more clearly define the conditions of consumption.

To identify the cause of low consistency, the weight coefficients of this indicator are re-determined with a discussion, and the coefficient of variation is calculated again. If after this procedure the coefficient of variation does not improve (or improves slightly), then the obtained values ​​of the weight coefficients should be discussed with the substantiation of the opinions of experts and the clarification of the consumer group.

The purpose of the discussions is to develop a common understanding among all experts of the nature of the work and issues and to identify the true opinion of each expert, and not to try to converge the values ​​of the weighting factors assigned by the experts. Thus, the coefficient of variation is, to a large extent, not a measure of “unfavorable” opinions, but a measure of the difference in the requirements of individual consumer groups for the product being evaluated.

9.3.3. sociological method

The sociological method for determining quality indicators is based on the collection and analysis of consumer opinions. Methods of collecting information differ depending on the purpose of the study. Sociological research includes four stages: research preparation; collection of primary sociological information; preparation of the collected information for processing and its processing; analysis of the processed information, preparation of a report on the results of the study, formulation of conclusions and recommendations.

The collection of consumer opinions is carried out in various ways:

• written survey (questionnaire);
• oral survey (interviewing);
• testing.

The survey is the most popular method of collecting sociological information. When conducting a survey, the source of information is the respondent, who is a direct participant in the survey.

Questioning is a written survey using a questionnaire. A sociological questionnaire is a set of questions united by a single goal and aimed at identifying the opinions of consumers. When compiling the questionnaire, it is very important to translate the language of specialists into the language of consumers. When compiling the questionnaire, the questions are formulated in closed, open and semi-open form.

Closed questions have all possible answers. For example:

"When do you prefer to rest?"

• 1 - in winter;
• 2 - in summer.

In semi-closed questions, the respondent is given the opportunity to complete the answer options. For example:

What dairy products do you prefer?

• 1 - kefir;
• 2 - yogurt;
• 3 - quark;
• 4 - others (please specify)_

Open questions do not contain answer options. For example:

What characteristics should a washing machine have?

The questionnaire may contain direct and indirect questions, main and control questions. Indirect questions are formulated in the case when the answer to a direct question requires the respondent to take a critical attitude towards himself, evaluate negative phenomena, and so on. The main questions are aimed at collecting information, and control questions allow you to assess the sincerity of the respondents. For example, if the main question is “What dietary supplements do you buy?”, then the control question in an open form could be: “What beneficial properties of dietary supplements do you know?”

Questions can be arranged in a row or in a table. When compiling a questionnaire, unambiguous understanding of the questions is essential.

In addition to the oral survey, questionnaires can be sent by mail. The advantage of the mail survey is the ease of organization, and the disadvantage is the incomplete return of questionnaires. Questioning can be conducted through the media (newspapers, magazines, television and radio broadcasting), it is based on the publication (broadcast) of the text of the questionnaire addressed to potential respondents. The advantage is the targeting of the audience, and the disadvantage is the low return of questionnaires. Internet surveys are conducted in various forms: mailing questionnaires by e-mail; placement of profiles in the news; Internet forums and teleconferences; questionnaires in the form of web pages, etc.

Interviewing involves an oral survey of respondents, which is conducted in a free form or in a standardized (formalized) form. An open-ended interview is often the initial stage in the development of a questionnaire or standardized interview and is conducted without a pre-prepared questionnaire. A standardized interview is conducted on well-defined questions. The advantage of the interview is the possibility of obtaining answers to all questions, clarification of controversial issues.

Testing is based on standardized questions and tasks that have a certain scale of values.

Tests allow you to get a quantitative estimate of the parameters, but require a certain preparation of the respondents.

The reliability of the results of collecting and processing information is ensured by the methods of mathematical statistics.

Sociological methods are widely used at the stage of marketing research, in the study of demand, to determine quality indicators, to assess quality, and so on.

For example, you need to find out what requirements an electric iron must meet. For this, a questionnaire is being developed indicating the parameters of the iron. Filling out the sheets is done by mail, when communicating with potential buyers at retail outlets, and so on. The results of the survey are shown in the table.

Results of a survey of potential buyers of irons

Options	Parameter values			Average score/
products				Number of responses
Weight, kg
power, kWt
Cord length, m
The presence of steam
Presence of Teflon coating
Type of heater
Winding the cord like a roulette
Heating rate
Appearance

The table shows the average ratings in points and the number of interviewed potential buyers who evaluated this parameter. The assessment was carried out on a 10-point scale.

To process the information received, you need to take into account the average score and the number of future buyers who voted for it. Then the sums of scores of each of the parameters and the total scores are determined. Next, the weight coefficients of each parameter are evaluated and the results are checked by summation:

q = (476/4444,8) + (342/4444,8) + (90/4444,8) + (403/4444,8) + + (486/4444,8) + (175/4444,8) + (216/4444,8) + (450/4444,8) + + (180/4444,8) + (480/4444,8) + (183/4444,8) + (320/4444,8) + + (497/4444,8) + (20,8/4444,8) + (126/4444,8) = 0,1071 + 0,0769 + + 0,0202 + 0,0906 + 0,1093 + 0,393 + 0,0485 + 0,1012 + 0,0405 + + 0,1079 + 0,0411 + 0,0719 + 0,1118 + 0,0047 + 0,0283 = 0,9998.

The sum of the weight coefficients is approximately one, which confirms the accuracy of the calculations.

1. The concept of the method of expert assessments

2. Application areas of expert evaluation methods

3. Stages of organization of expertise

4. Requirements for experts

1. Method of expert assessments- method analysis and generalization of judgments and guesses with experts. This method is used when rational mathematical methods are ineffective in solving problems. An intuitive-logical analysis of the problem is carried out, followed by a quantitative assessment of judgments and formal processing of the results.

The problems to be solved can be conditionally divided:

To problems provided with information;

Problems for which there is a lack of information. These methods are especially effective in conditions:

Great uncertainty of the environment in which the object under study operates;

Lack of time and in extreme situations;

Lack of a reliable theoretical basis.

2. Methods of expert assessment are used in solving the following problems:

Compilation of a list of possible events in various areas for a certain period of time;

Determining the most probable time intervals for a set of events;

Definition of goals and objectives of management with their ordering in order of importance;

Definition of alternative options for solving problems with an assessment of their preference;

Alternative distribution of resources for solving problems with an assessment of their preference;

Alternative decision-making options in a certain situation with an assessment of their preference.

3. The process of organizing an examination can be divided into the following stages:

drafting a guidance document. It specifies the main provisions of the examination:

Tasks for its implementation;

Composition and responsibilities of the working group and the expert group;

Resources required to ensure the work, and the timing of the work;

selection of working and expert groups. Two groups take part in the examination:

working group, which consists of an organizer, a system engineer and a technical worker;

. expert group, i.e. a group whose expert opinions will form the basis for making future decisions;

development of a methodology for conducting a survey (if necessary). At this stage, the following are determined: the place and time of the survey; tasks; form of holding; procedure for fixing and collecting results; composition of the required documents. Depending on the time available for the examination, the complexity of the problem under consideration, the participating specialists, the following can be distinguished survey forms:

. individual- the abilities and knowledge of each specialist are used to the maximum;

. group (collective)- allows experts to exchange opinions and, on their basis, adjust their assessment. But with this method of questioning, a strong influence of authorities on specialists can appear;

. personal (full-time)- the survey is carried out with direct contact, for example, the interviewer and the expert;

. correspondence- one common example of this survey method is the mailing of questionnaires. There is no direct interaction between the interviewer and the expert;

. oral- for example, an interview;

writing- for example, filling out a questionnaire;

open- the survey process and its results can be studied by other experts;

hidden- expressed ideas and decisions of experts are secret;

development of survey data processing methodology. Processing of expert survey data can be done in 2 ways:

Checking the consistency of the opinions of experts (or classification of experts, if there is no consistency);

. averaging the opinions of experts within the agreed group;

presentation of work results. An analysis of the results obtained as a result of the examination is carried out, on the basis of which a report is drawn up. After discussion and approval of the results obtained, the results of the work done are provided to the customers of the expertise.

4. When forming the expert and working groups, it is necessary to present the following requirements for experts:

Estimates put forward by experts should be stable over time;

With the introduction of additional information, the expert assessment should improve, but it should not fundamentally change from the originally formulated assessment;

The expert must be a recognized specialist in the field of knowledge under study;

The expert must have experience of participating in such examinations;

Expert opinions must be stable. Sustainability determined by the nature of the errors that experts can make during the examination . In this regard, there are 2 types of errors:

systematic, which are characterized by a stable positive or negative deviation from the true value;

. random when the output values ​​of the experts are characterized by a large dispersion.

Synectics as a method for studying control systems

1. The concept of "synectics"

2. Features of the synectics method

3. Stages of the synectics method

4. synectics operators

1. Synectics(translated from Greek) is combination of dissimilar and sometimes even incompatible elements . The synectics method method of finding new solutions proposed W. Gordon in the USA in 1961. in his book "Synectics: development of creative imagination" in order to increase the likelihood of success in the process of setting and solving problems.

The main idea of ​​this method is that during creative activity when creating special conditions, a person puts forward unexpected analogies and associations regarding the problem under study. Creative activity is understood as mental activity in the process of solving problems, the result of which is an artistic or technical discovery. In other words, a person comes to a decision by activation of unconscious mechanisms in the process of conscious exploration of problems. The idea of ​​the "synectics" method as a method of studying control systems is to create a special, constant "groups of synectors" (5-7 people) to identify, formulate and solve problems that arise during the operation of control systems.

2. The synectics method has the following features:

decision making approach is that the idea put forward is a complete, integral thought, the author of which is the individual who expressed it. A holistic idea can be accepted by others or rejected, but no one else has the right to its authorship. But other members of the group may come to a different thought on the basis of this idea, the expression of which may lead to the manifestation in the memory of metaphors, patterns, vague sensations that act on the feelings and intuition of a person;

creative activity of synectors it is ensured that the group influences the creative activity of each synector in such a way that in the process of putting forward new ideas, individuals try to surpass themselves, use non-standard decision-making approaches and take on the greatest part of the difficulties;

selection of group members carried out taking into account the emotional type of the individual, which is determined by the following characteristics:

Is he trying to get to the bottom of the problem right away, or is he beating around the bush;

Whether he behaves passively in the face of imminent defeat, or persistently strives to succeed;

When he is wrong, whether he connects it with his actions or justifies himself, he looks for reasons outside;

. whether he can use his intellectual energy effectively in difficult situations or gives up at the most critical moment;

Devil's Advocate: due to the fact that the "group of synectors" consists of professionals from various fields of knowledge in order to study the problem from various points of view, it cannot know everything. That's why depending on the nature of the problem under study, it includes a "devil's advocate" - an expert in the field of the problem under study. Its functions include:

Translation of the specific terminology of your field of knowledge into the public domain;

. translation of the ideas put forward into the terminology of their specialty;

. disclosure and rejection of the weaknesses of the proposed approaches;

Disclosure of the features of his area of ​​expertise to the group;

implementation of the ideas put forward: the ideas put forward by the synectors in the process of studying the problem must be practically implemented. Synectors are also involved in the process of implementing the ideas they put forward.

3. The synectics method consists of the following steps:

Formulation of the problem;

Translation of the task, "as it is stated", into the task, "as it is understood";

Identification of a question that causes analogies;

Work on finding analogies;

use of analogies, among which:

Direct analogy;

. symbolic analogy;

. personal analogy;

. fantastic analogy;

Search for the possibilities of translating the found analogies and images into proposals for solving the problem.

4. Operators of synectics- specific psychological factors that contribute to the creative process, they are designed to enhance involvement, empathy, play, etc.

Synectics consists of two main processes:

turning the unknown into the known: the solution of each problem begins with its understanding, i.e., a new problem is reduced to an already known one in order to reduce its threat to a person (every new thing threatens a person due to his conservatism). Initially, the explanation of the new problem is given within the framework of the familiar model;

turning the known into the unknown: the known is rethought, distorted depending on the new problem. In our everyday life, the same objects appear differently to different people. The transformation of the known into the unknown is carried out using 4 mechanisms:

personal analogy- personal identification of the researcher with the elements of the problem, for which the researcher needs to have creative imagination and the art of research activity. Such an analogy helps to see previously unknown aspects of the problem;

direct analogy- transfer of comparison parameters from one object to another from parallel existing areas of knowledge and technologies. Direct analogy requires the researcher to have education, multifaceted knowledge, and various profile training;

symbolic analogy- based on the description of the problem using objective and non-personal images, i.e. the essence of the problem or its solution is described by a brief metaphor, which, on the one hand, is a characteristic of the problem, and on the other hand, a contradiction. For example, an exhibition is an organized accident; soft drink - transparent ghost; the book is a silent interlocutor;

fantastic analogy- uses freedom when using symbols. By using a fantastic analogy, a researcher can see an unexpected solution to a problem that cannot be found in the real world due to the existence of contradictory objective laws.

Delphi Method

1. Definition of the concept of the "Delphi" method

2. Stages of the "Delphi" method

3. Advantages of the Delphi method

1. Method "Delphi"- one of the methods of expert assessments, with the help of which quick search for solutions, among which the best is selected . Its other name is the "Delphic oracle", which he received in ancient Greece. This method was developed O. Helmer and his colleagues, it was originally created for the purpose of scientific and technical forecasting of the future. It is based on the principle of increasing the level of reliability of information received from a group of experts, i.e., collective expert assessments.

Methods of expert assessments in forecasting and long-term planning of scientific and technological progress are used in the following cases:

a) in the absence of sufficiently representative and reliable statistics on the characteristics of the object (for example, lasers, holographic storage devices, rational use of water resources in enterprises);

b) under conditions of great uncertainty in the environment for the functioning of an object (for example, forecasts of a man-machine system in space or taking into account the mutual influence of the fields of science and technology);

c) in medium- and long-term forecasting of objects of new industries that are strongly influenced by new discoveries in the fundamental sciences (for example, the microbiological industry, quantum electronics, nuclear engineering);

d) in conditions of lack of time or extreme situations.

An expert assessment is necessary when there is no proper theoretical basis for the development of an object. The degree of reliability of expertise is established by the absolute frequency with which the expert's assessment is ultimately confirmed by subsequent events. There are two categories of experts - these are narrow specialists and generalists who provide the formulation of large problems and the construction of models. The choice of experts for the forecast is made on the basis of their reputation among a certain category of specialists. However, one should not forget the circumstance that a first-class specialist cannot always adequately consider and understand general, global issues. For this purpose, it is necessary to involve experts, although not narrowly informed, but possessing the ability to be daring and imaginative.

"Expert" in literal translation from Latin means "experienced". Therefore, both in the formalized and non-formalized ways of defining an expert, professional experience and intuition developed on its basis occupy a significant place. The conditions for the necessity and sufficiency of referring a specialist to the category of experts are introduced as follows.

It is important to establish not the absolute degree of reliability of the expert assessment, but the degree of reliability in comparison with the assessment of the average specialist, as well as the correlation between the probability of his predictive assessment and the reliability of the class of hypotheses that the expert operates with. In general, you need to define what an expert is. Here are some of the requirements that an expert must meet:

1) expert estimates must be stable in time and transitive; 2) the availability of additional information about the predicted signs only improves the assessment of the expert; 3) the expert must be a recognized specialist in this field of knowledge; 4) the expert must have some experience of successful forecasts in the given field of knowledge.

When characterizing experts, it should be borne in mind that two types of errors may occur as a result of the development of estimates. Errors of the first type are known in measurement technology as systematic, errors of the second type as random. An EA that is prone to errors of the first type produces values ​​that steadily differ from the true one in the direction of increasing or decreasing. Errors of this kind are believed to be due to the mentality of the experts. To correct systematic errors, you can apply correction factors or use specially designed training games. Errors of the second type are characterized by the magnitude of the dispersion. Based on the analysis of the main types of errors in making expert judgments, one more thing can be added to the list of requirements for experts considered earlier. Its meaning is that one should prefer an expert, whose estimates have a small variance and a systematic deviation of the mean error from zero, to an expert with a mean error equal to zero, but with a larger variance. Unfortunately, it is impossible to determine a priori the ability of a person to make correct expert assessments. An important means of preparing experts are special training games.

The organization of the expert’s work forms can be programmed or non-programmed, and the expert’s activities can be carried out orally (interviews) or in writing (answering questions from special tables of expert assessments or a free presentation on a given topic).

Programming the form of work of an Expert Advisor involves:

building a graph model of an object based on a retrospective analysis; determination of the structure of the tables of expert assessments (feasibility study) or the interview program based on the graph model of the object and the goals of the examination; determining the type and form of questions in a feasibility study or in an interview;

determining the type of scale for questions in the feasibility study; taking into account the psychological characteristics of the examination in determining the sequence of questions in the feasibility study; accounting for verification questions; development of logical methods for the subsequent synthesis of predictive estimates in complex forecasts of the object.

The organization of stimulation of the work of an expert consists in the development of:

heuristic techniques and methods that facilitate the search for a predictive expert assessment; legal norms that guarantee the expert registration of priority and authorship, as well as non-disclosure of all scientific and technical ideas put forward by him in the process of examination;

forms of moral, professional and material interest of an expert in expert assessments; organizational forms of the expert's work (inclusion in the work plan, etc.).

Based on the model of the forecasting object obtained as a result of the analysis, the scientific and technical areas in which it is necessary to involve an expert are determined, groups of experts are distinguished according to whether the issue belongs to the field of fundamental, applied sciences or to joint scientific areas.

When solving the problem of forming an expert group, it is necessary to identify and stabilize an efficient network of experts. The way to stabilize the expert network is as follows. Based on the analysis of the literature on the predicted problem, any specialist with several publications in this field is selected. He is asked to name the 10 most competent, in his opinion, experts on this problem. Then they turn simultaneously to each of the ten named specialists with a request to indicate the 10 most prominent of their fellow scientists. From the list of specialists received, 10 initial ones are deleted, and letters containing the above request are sent to the rest. This procedure is repeated until none of the newly named specialists adds new names to the list of experts, i.e. until the network of experts stabilizes. The resulting network of experts can be considered a general set of specialists who are competent in the field of the predicted problem. However, due to a number of practical limitations, it turns out to be inappropriate to involve all specialists in the examination. Therefore, it is necessary to form a representative sample from the general population of experts.

The determination of the specifics of procedures for PEO class methods (personal expert assessments) is carried out on the basis of an analysis of the requirements for experts and their assessments arising from the essence of the methods:

a) analytical notes make requirements for structuring the problem being experimented with, explication and ranking of goals, analysis of alternative ways to achieve the goal, cost estimates for each alternative, and recommendations on the most effective ways to solve problems;

b) paired comparisons, normalization and ranking require the homogeneity of the evaluated features, the presence of logically justified criteria and standards, the presence of unambiguously defined procedures for operating with criteria, standards and features;

c) interview impose specific requirements on both the expert and the interviewer;

d) morphological structuring requires a clear definition of the functional characteristics of the object or problem that needs to be improved, the classification of scientific principles, on the basis of which it is possible to improve the characteristics; analysis of all possible combinations of these principles and elimination of obviously absurd ones; assessment of combinations according to the degree of feasibility and the cost of their implementation; comparison of combinations according to the complex criterion "costs - efficiency - time".