Use feedback scientifically through questionnaires and polls!

Use feedback scientificallythrough questionnaires and polls!

From: Claudia Wilimzig, Nandana Nielsen, Virginie Vernois, Christian Hanisch, Evgeny Zolotarev, Karl Nielsen, Leo Angart, Philippe Vernois, Viacheslav Balyberdin

In January 2018 about 80 of the 150 NLP Trainer from the NLP Leadership Summit met for 3 days the second time (first time was 2016) in Alicante (Spain). This is a group of highly experienced NLP Master Trainer from all over the world who have more than 15 years of NLP training experience.

The book "Powered by NLP 2" was produced as a result shortly after this meeting. You may find it as free download on the website of the NLP Leadership Summit, in the right side bar.

On page 99 to 118 you may find the contribution of Claudia Wilimzig, Nandana Nielsen, Virginie Vernois, Christian Hanisch, Evgeny Zolotarev, Karl Nielsen, Leo Angart, Philippe Vernois and Viacheslav Balyberdin about: "Use feedback scientifically through questionnaires and polls!" Virginie, Philippe, Christian, Leo and Karl were in Alicante and worked during and after the NLP Leadership Summit on this topic. Nandana, Claudia, Evgeny and Viacheslav belong to this group as well and worked before and after Alicante on this topic, but they were not present in Alicante.

These 9 NLP experts write in this part of the book: "Powered by NLP 2" about:

• The general importance of feedback
• Questionnaires for NLP Master Trainers regarding the expectations and realized benefits of their seminar participants
• Doctoral Candidates use the poll from the NLP Leadership Summit regarding: “What is NLP?” for further research

Their main recommendation is: Use scientific feedback methods to capture, present and communicate your successes in a structured and evidence-based way.

The whole meeting of the NLP Leadership Summit in Alicante was a great feedback process where highly experienced leader of the NLP field listened to each other and thus learned from each other. Feedback is strongly connected to the whole concept of NLP. So we think that using a questionnaire and a poll for feedback fits pretty well to the NLP approach

During the NLP Leadership Summit and the day after Christian, Philippe, Virginie, Karl and Leo (from left to right) worked in Alicante on the development of a feedback questionnaire for scientifically researching the expectations and benefits of seminar participants regarding NLP seminars. We dreamed about what we could write on our websites as scientifically researched benefits of our trainings. We asked each other what our participants keep telling us about how our trainings supported them to realize their goals and to enjoy life. We had so many inspiring examples from our daily experience. Wouldn’t it be wonderful to document this scientifically through a proper feedback process using questionnaires? You find the fruits of our work in chapter 2. Virginie and Philippe from France write their dissertation with these questionnaires at Universidad Central de Nicaragua (UCN) that offers at their Psychology Faculty Psychology with a focus in NLP.

Two other Doctoral Candidates of the UCN, Evgeny (Eugen) and Viacheslav (Slava) from Russia, use the poll about “What is NLP?”, that was conducted by the members of the NLP Leadership Summit, as a basis for researching this topic more deeply by asking the NLP Master Trainers from the International Association of NLP Institutes and including the perspective of the more than 3.400 members of this association from 63 countries. They describe their approach on chapter 3 here.

Claudia, Nandana, Christian and Karl are the Professors at the UCN for the 4 Doctoral Candidates.

 „I didn’t know what I had said before I heard the response of my communication partner. “ (Paul Watzlawick)

Everybody constantly receives feedback. Almost every response is feedback, be it verbal or nonverbal. It is not possible not to communicate (the famous psychologist Paul Watzlawick). It is also not possible not to receive feedback – every act of communication includes feedback - but a lot of people do not want to perceive feedback and do not value it. Or they simply ignore it and refuse to think about it. Feedback is a good and important method to get to know oneself. To get to learn yourself you „only“ have to listen to feedback – and modern day psychology provides a lot of tools for integrating the perception of feedback into professional as well as private life. In ancient times people went to the desert or into other places of solitude to get to know themselves and to gain new insights. There are easier ways and ways more practical to integrate into everyday life to develop yourself.

Due to its strong orientation towards individual development of oneself client’s feedback has always played an important role in NLP. It is important not to see one’s own opinion and concepts how life should be as „given“ but to constantly question them. NLP professional consider feedback as a chance whether what they think and what they want to communicate is understood by the receiver the same way. The development of perception is one of the pillars of NLP. This provides a professional tool to understand whether there is a discrepancy between the intention of the practitioner/trainer and the reception of the recipient and to enable him to act upon it.

Milton Erickson said something along the gist of „There are no errors, only feedback“. This is one of the presuppositions of NLP. One should be careful how to interpret this quote. „Error“ is perceived as a lack or failure. This pattern of interpretation is unproductive.

There are a few rules regarding feedback one should implement and seen as given. Feedback should be perceived as constructive and one should concentrate on this aspect of the feedback given. As already said there always is feedback implemented in every conversation which you should regard from your educated level of perception and if applicable analyze using scientific methods. You should make room for integrating analysis of feedback into your practice. Feedback should be received as descriptive rather than judging! Feedback should be perceived as criticizing behavior not the person himself.

Several fields should seriously consider feedback – but to an astonishing and alarming degree they actually don’t. One important field is psychotherapy. A large number of psychotherapists don’t believe in systematically analyzing their interventions with feedback – one argument being that one-to-one interactions are so individual and so unique that studies leading to so-called evidence-based psychotherapy are not possible. First of all, feedback should not be equalized with large-scale standardized studies. Feedback and its scientific analysis and evaluation is possible even for individual set-ups. Second, negative attitudes have to do more with a disregard for the value of feedback. Alarmingly, especially in psychotherapy once one establishes feedbacks one often finds out that the outcome is not what the therapists wants it to be and what he thinks to be natural and self-evident. Researchers like Scott Miller ( www.scottdmiller.com ) have made it their mission to establish a new spirit of integrating feedback into the development of psychotherapy. Our questionnaires about the benefits of NLP (chapter 2) and “What is NLP?” (chapter 3) highly support this scientific approach for NLP.

These questionnaires are designed through expert opinion in order to give you profound feedback at the beginning about the expectations of your participants and at the end about the results of your training. To use such a feedback instrument is highly professional. It opens for your participants at the beginning the door to additional expectations and it sends the messages that your trainings have a huge impact for great professional and life benefits and that they are scientifically designed and controlled. This adds value to your training and you can use it for your marketing.

The scientific design of the following questionnaires is based on expert opinion. They were developed in several steps. The main step was in Alicante on the day after the NLP Leadership Summit.

• A commission of 12 IN and ICI Master Trainer discussed many years questionnaires that other scientists used for researching the benefits of trainings.
• Claudia and Karl from this commission met for 4 days in December 2017 in Berlin for the first draft of a questionnaire for measuring the benefits of NLP trainings on the background of the work of the commission and that Claudia was exploring websites for the claims of positive effects of NLP trainings.
• In Alicante Karl interviewed on January 15th 2018, 4 highly experienced experts of NLP trainings (Virginie, Christian, Leo, Philippe) for several hours regarding their hypothesizes about the benefits of the results of NLP trainings and how these hypothesizes could be expressed in easy to understand questions.
• On January 19th 2018 the 4 experts from Alicante plus 10 more highly experienced experts in the field were asked about their feedback regarding the questionnaire that we developed on the basis of the data from Alicante.
• On the basis of this feedback this questionnaire was further developed and given back to the 14 experts for confirmation on January 20th 2018.
• On January 24th 2018 the IN sent the further developed questionnaire to 166 highly qualified experts (the seal entitled Master Trainer in IN ICI WHO) and asked them to do as much pre-testing as possible
• In February 2018 we received from 10 Master Trainers feedback and from 2 Master Trainers filled out questionnaires. With this feedback we further improved the questionnaire.
• Now we are still in the pre-test phase for the rest of 2018. In 2019 we plan to use the feedback from expected more than 1.000 questionnaires for designing a version that includes only the highest voted questions. With this version we will highly recommend to all our IN ICI WHO Master Trainers to use it. constantly.

Besides the version for IN ICI WHO members we have developed the following version that can be used from other associations as well. They are online here:

www.in-ici.net/PDFs/open-Questionnaire-Page-for-Mastertrainer.pdf

www.in-ici.net/PDFs/open-Questionnaire-Expectations-Beginn-of-Training.pdf

www.in-ici.net/PDFs/open-Questionnaire-Benefits-End-of-Training.pdf

Designing the online form of the poll, considerations for the selection of elements of the voting list, engaging unconscious motivation of the respondents

The subject of this chapter of the article are the issues relevant to the creation of the online questionnaire which would allow to resolve different classes of problems relevant to NLP (including curricula determination) by a process of voting.

This questionnaire could be applied to the task of determining which technique is NLP. In order to do so, we should formulate the voting alternative. One of such alternative is “X is a part of NLP” or “X is not a part of NLP”, where X is an NLP technique.

For the purpose of further argument, it is reasonable to bear in mind two categories of respondents: expert-level NLP leaders (which we will call respondents A) and regular NLP users (respondents B). Brief overview of main issues of questionnaire design and development process follows.

3.1 Objectiveness of online research

From a researcher’s point of view the question of objectiveness of gathered data must be considered first.

Firstly, will the unconscious be engaged in the answers of respondents? Will the respondents be motivated to answer fully and honestly, not just formally?

Secondly, from analyzing and developing the NLP concept of “signal systems” it could be deduced that there is no such thing as binary “yes or no” signal system. There are always at least four gradations of signals: “yes”, “yes (with internal meaning of “no”)”, “no (with internal meaning of “yes”)”, “no”. There is always a number of techniques with respect to which a particular expert is fully congruent, whether he/she thinks that it is part of NLP or not. But there is always a number of techniques with respect to which the same expert is ambivalent. In any context where there is such kind of ambivalence with respect to complex decisions, signals “yes (with internal meaning of “no”)” and “no (with internal meaning of “yes”)” appear. Therefore, there is a need to take into account conscious and unconscious evaluations, motivation and emotions, etc.

Thirdly, the question of objectiveness of online research is a big question for social sciences, and thus should be addressed properly.

From the point of view of the field of NLP, that question could be addressed in the following way: objective answers are received when rational decision-making process of experts’ minds are supported by the rapport with their unconscious.

While in online survey it is technically impossible to utilize most conventional rapport-building and unconscious motivation methods, there is still a place for some of the classical ways to establish rapport with the unconscious.

Let us consider some of them.

3.1.1. Literalism of the first code of NLP

As it is known from NLP literature, the unconscious perceives and expresses verbal content in a literal manner. For example, as is written in one of the first NLP books, “The Structure of Magic”:

Most people, in describing their experiences, even in casual conversation, are quite literal. (Bandler & Grinder, 1976, p. 11)

One of the examples of literalness is NLP “quotes pattern”. As J. Grinder describes it:

People have almost no consciousness of any meta-levels if you distract them with content. Once at a conference I talked to a large group of psychologists who were pretty stuffy and asked a lot of dumb questions. I told them about quotes as a pattern. Then I said for example—I even told them what I was doing—Milton Erickson once told me a story about a time he stayed at a turkey farm, and the turkeys made a lot of noise and kept him awake at night. He didn't know what to do. So finally one night he walked outside—and I faced all those psychologists out there—and he realized he was surrounded by turkeys, hundreds of turkeys everywhere. Turkeys here, and turkeys there, and turkeys all over the place. And he looked at them and he said “You turkeys!”

There were a couple of people there who knew what I was doing and they absolutely cracked up.

(Bandler & Grinder, 1979, p. 62)

Authors hope that the readers understand in which sense this quote is to be taken — as a regulatory “stop-anchor” for optimization of important discussions over complex topics.

That tradition of literalism is in congruence with Erickson’s approach to communication:

The literalness of the trance state causes the patient to have a new pattern of listening. He listens to the words in the trance state rather than to the ideas. (Erickson & Rossi, 1973, p. 1)

Trance phenomena won’t be observed in online questionnaire, but literalness of the unconscious perception necessarily will be present in every case. On a conscious level, especially for respondents A, voting elements of different logical types like “Life and mind are systemic process” or “Submodalities” literally make sense without any additional specification — and for respondents B it is necessary to specify what it literally means, and ensure that each respondent actually votes for the same thing (same class of underlying VAKOG).

Simply saying, a literal and specific description of what is what should necessarily be attached to the voting list, in order to maintain rapport with the unconscious, which processes information in a literal and specific manner. Specific application of the idea of literalism to the survey is the way in which techniques should be described, which follows.

3.1.2. Literal-systemic reference sources of technique description

In the subtitle it is argued that in the questionnaire, each technique should have hyperlink to online or other (e.g. literary) source of its literal-systemic description.

Brilliant examples of such descriptions are to be found in the trinity of first code NLP books, “Frogs into Princes” (Bandler & Grinder, 1979), “Trance-formations” (Grinder & Bandler, 1981) and “Reframing” (Bandler & Grinder, 1982). Editors of these books were S. & C. Andreas, and their books are following the same structure as well, e.g. “Change your mind—and keep the change” (S. Andreas & Andreas, 1987) and “Heart of the mind” (C. Andreas & Andreas, 1989).

So, in the aforementioned sources NLP techniques are described using the following three required elements:

• Elaborate preamble — “presupposition vortex”.a) First general/overview demonstration of a technique with minimal comments. There is nothing to be added to that item.b) The complex of concepts which gradually concentrates into a number of ideas which create anticipation of a technique.c) In case of a complex technique, presupposition vortex, which leads to it, consists of descriptions of simpler techniques which are a part of technique in question. E.g. six-step reframing in “Frogs into Princes” is described lastly, but each previous technique in the book serves as a premise for the main technique.
• Step-by-step algorithm.

  In the texts of source books, step-by-step plans of each technique correspond to this item.

• Three levels of variation of technique application.a) Descriptions of how a technique could be applied to different people, in a form of answers to students’ questions.b) Discussions of a differential diversity of experiences with the technique in training group.c) Extended discussion on technique variations drawing from a broad therapy content (experience of other therapists, life cases, etc.), technique variations, modifications and derivations.

There are several lists of NLP techniques available in the NLP community. One example of an elaborate list of NLP techniques available in the NLP literature is the article “Techniques” from R. Dilts’ “Encyclopedia of Systemic NLP…” (Dilts & DeLozier, 2000, pp. 1396–1398). 31 techniques are listed. (Authors express their gratitude for this valuable comment to A. Durlinger.)

From the technical standpoint, there is a task of development of the user interface, whether to include all three content elements of each technique in the voting list itself or just have a hyperlink to complete descriptions (or just refer to the name of a source book)? One way is to present a name of a technique and steps (i.e. an algorithm) embedded into the list and have other two elements (demonstration in two contexts and manual for dealing with obstacles) hyperlinked. Another alternative would be to include demonstrations (in form of text or multimedia format) into the list itself: the content of such elements (explanation, steps, demonstrations) can be minimized to a clickable text to be unfolded after clicking it without the need to proceed to another web page.

3.2. Subjective choice for alternatives to support motivation

Let us consider a typical online survey. One thing to notice is that an introduction to the survey (instructions and main survey question) and survey elements (in our case, NLP techniques) are literally separated by some amount of space. Hence, there is a difference between what the researcher meant (and expressed in his or her instructions and introduction) and what the respondents would mean when they put their own check mark at the position which they selected.

Simply saying, respondents imply their own meaning when they put their mark, which may or may not correspond to what the researcher meant. Check marks are not just marks, they are, strictly speaking, non-verbal expressions of a respondent’s mind with a particular subjective individual meaning attached to them. In order to increase precision of the gathered data, let us consider the following idea of increasing subjective freedom of choice.

Base alternative of the survey is “This is a part of NLP vs. This is not a part of NLP”. I.e., respondents will look through the list of techniques marking them with one part of alternative (“X is part of NLP”) or another (“X is not part of NLP”).

While this strictness eases the data analysis, it doesn’t help to increase resources of respondents’ motivation and trustworthiness of their answers. As Oxford Internet Institute research tells, “where a physically present interviewer could gently motivate somebody to continue filling out a survey, this is much harder to do online” (Wiersma, 2013, pp. 5–6).

In order to increase unconscious motivation and precision of the answers, it is necessary to give another dimension of subjective choice to the participants. The general idea of that maneuver comes from the Ericksonian approach:

I give him a feeling of choice even though I’m determining it (Erickson & Havens, 1996, p. 110)

So, each respondent should be able to actually select one’s own variation of the base alternative (“X is a part of NLP vs. X is not a part of NLP”).

Possible dimensions for such alternative are the following:

• There is a set of basic psychological referential alternatives, namely: “yes or no”, “like or dislike”, “significant or insignificant”, “important or not important”. These minimal set of alternatives covers conscious and subconscious (left- and right-hemisphere) modes of reference.
• There is a set of content alternatives: NLP definition, NLP development, NLP education, NLP therapy, NLP applications.
• There is a set of respondents’ positions of reference: I, you, he/she, them, etc.

So, possible alternatives could be yielded by combining the dimensions above, e.g.:

• This technique is a part of NLP from my perspective vs. This technique part of NLP from someone else’s perspective.
• I like that technique vs. I dislike that technique.
• This technique is significant for NLP trainings vs. This technique is significant for NLP development.
• etc.

For an online survey full list of theoretically possible alternatives should be reduced to practical list of 5±2 alternatives.

So, a respondent will be able to select literal wording of the alternative which he or she would like to use to evaluate a list of techniques.

Then there are additional options of survey design fine-tuning. Should respondents be allowed to evaluate only a number of techniques (which is selected by themselves), or be instructed to evaluate the whole list? Should they be allowed to change subjective alternative in the course of the survey (i.e. to evaluate each technique with a separate alternative), or be instructed to go with one selected alternative for every selected technique?

Another user interface question is the following: in order to reduce clutter each technique should be shown at separate page. That would allow more information to be displayed. But, at the same time, a list of all techniques (or ones which are pre-selected by a respondent) should be readily available (ideally, constantly present before a respondent’s eyes) in order to serve as a subconscious anchor to produce integral (internally congruent) results/answers for each technique.

3.3. Formal model of choice between alternatives

Every form of such kind of research is revolving around the concept of alternatives (two polarities of reference according to which content is evaluated), and it is necessary to mention that the specific NLP model could be proposed, which allows deep study of such processes and their content.

There is a wide array of scientific premises to such model, including the studies of H. Bergson (Bergson, 2001) on the choice process, A. Maslow’s works on motivation (Maslow, 1943), modern-day L. Vygotsky’s school of psychology with a concept of functional levels of “dynamic regulatory systems (DRS)” (Kornilova, 2005).

From the practical standpoint, there is the NLP technique “Shifting the criteria of importance” (S. Andreas & Andreas, 1987, pp. 61–86), which outlines the process of eliciting a “hierarchy of criteria” for some daily problem (e.g. “to wash dishes vs. not to wash dishes”).

Fig.1. Value hierarchy interview plan for “X belongs to NLP vs. X doesn’t belong to NLP” alternative

Fig. 2. Dual regulation of behavior (VAKOG and value) in the Value Hierarchy model

Analyzing the theoretical background and further developing practical methods we could hypothesize that “hierarchies of criteria”, “dynamic regulatory systems”, and “value hierarchies” are different facets of the same underlying psychological processes, which regulate any “evaluation”, “motivation” and “decision-making”, be it pragmatic behavioral choice between washing or not washing dishes, or some complex ethical or cognitive evaluation like Shakespearean “to be or not to be”.

Thus, we could propose a survey for the main research question “What techniques are part of NLP” based on the Value Hierarchy model, according to the sensory experience and internal values of the respondents. We could only briefly summarize (Fig. 1) the “matrix” (array of interview questions — for one example alternative of “X belongs to NLP vs. X doesn’t belong to NLP”) for this kind of research here, as full description is out of scope of this article. Fig.2 depicts the main idea/principle of an individual functional level of value hierarchy — dual regulation of activity.

An online survey which would follow such plan requires complex design and development.

3.4. Additional applications for online questionnaire

The overview above pointed out the general issues which should be resolved during the process of development of the online questionnaire in order for it to be used to determine answers to general NLP questions, e.g. which technique is part of NLP and which is not. It is worth noting that this question is not the single one which could be formulated for an online survey. Very brief overview of additional applications of the survey follows.

3.4.1. Length and content of NLP training seminars

Currently, length and content of NLP training seminars for several NLP associations — including and not limited to NLP IN (International Association of NLP Institutes (IN), 2017), IA NLP (International Association For Neuro-Linguistic Programming, 2017), INLPTA (International NLP Trainers Association, 2005) — is determined by guidelines which include recommended total length of specific qualification levels and recommended content & skills which an NLP specialist of a particular level must have attained.

It would be interesting, in order to harmonize standards across the global NLP community and have a deeper understanding of optimal curricula length and content, to investigate what technique (from the same list as from the main question of this article, “what is part of NLP”) requires what educational time. Where educational time may include not only face-to-face trainings, but self-education (in a form of online & book reading) and self-training (in contexts of one’s daily life or self-organized supervised groups). Such research, to our knowledge, has not been done yet, and not only it can create common ground for different associations to base their standards on, but will also allow well-grounded recommendations for “blended learning” (partly online/remote) NLP education programs.

Blended and online education, no doubt, is one of the leading trends of modern-day education in general, but in the field of NLP education that topic raises, due to the known objective and subjective reasons, controversy and is debatable.

3.4.2. Embedded discussion platform

In order to gather opinions of both groups of respondents, A and B, it would be beneficial to not only receive feedback by conventional means (e.g. emails sent directly to the authors), but to be able to discuss content of the list, including the suggestions for new list elements or for improvement (i.e. adding or referencing elements of techniques which are required for complete description - according to the previous sections of this article).

For such discussions to be convenient, from a technical standpoint, a transparent authorization mechanism should be developed — to allow respondents who were invited to participate in the survey by their associations and who are at the same time logged in into their associations’ websites to be able to seamlessly log in to the discussion platform.

3.5. Summary of chapter 3

Issues which are relevant to the creation of an online platform, which would allow to solve several classes of problems important for modern NLP, have been overviewed in this article. The list of three required elements (“presupposition vortex”, step-by-step plan, technique variations) which are used to introduce and describe techniques in recognized first code NLP books is provided.

The question of selecting alternatives for the questionnaire is formulated, several options are considered, and the formal model for choice process (including cognitive choice) is introduced briefly.

Additional applications of online survey, such as gathering data for harmonization of NLP educational programs are outlined.

Literature

Andreas, C., & Andreas, S. (1989). Heart of the mind. Real People Press.

Andreas, S., & Andreas, C. (1987). Change your mind—and keep the change: advanced NLP submodalities interventions. Real People Press.

Bandler, R., & Grinder, J. (1976). The Structure of Magic II. Palo Alto, California: Science and Behavior Books, Inc.

Bandler, R., & Grinder, J. (1979). Frogs into princes: Neuro Linguistic Programming. (S. Andreas, Ed.). Real People Press.

Bandler, R., & Grinder, J. (1982). Reframing: neuro-linguistic programming and the transformation of meaning. (S. Andreas & C. Andreas, Eds.). Real People Press.

Bergson, H. (2001). Time and free will: An essay on the immediate data of consciousness. Courier Corporation.

Dilts, R., & DeLozier, J. (2000). Encyclopedia of systemic neuro-linguistic programming and NLP new coding. Retrieved March 13, 2018, from http://nlpuniversitypress.com/

Erickson, M., & Havens, R. (1996). The wisdom of Milton H. Erickson (Irvington). NY.

Erickson, M., & Rossi, E. (1973). Literalness and the use of trance in neurosis. Dialogue between Milton H. Erickson and Ernest L. Rossi. In Milton H. Erickson, M.D.: Complete Works. The Milton H. Erickson Foundation.

Grinder, J., & Bandler, R. (1981). Trance-formations. Neuro-linguistic programming and the structure of hypnosis. (C. Andreas, Ed.). Real People Press.

International Association for Neuro-Linguistic Programming. (2017). Criteria for Recognition as Fellow Member Trainer IANLP. Retrieved February 5, 2017, from https://www.ia-nlp.org/web/curricula/section/Criteria_of_Recognition_as_Fellow_Member_Trainer_IANLP

International Association of NLP Institutes (IN). (2017). IN Curricula. Retrieved May 5, 2017, from https://www.nlp-institutes.net/about/curricula.php

International NLP Trainers Association. (2005). INLPTA Standards. Retrieved March 1, 2018, from http://www.inlpta.org/downloads/pdf/inlpta_nlp_diploma.pdf

Kornilova, T. V. (2005). Methodological problems of decision making psychology. Psychological Journal, 26(1), 7–17.

Maslow, A. (1943). A Theory of Human Motivation. Psychological Review, 50, 370–396.