Summer Break!

20151104_144422Hello all!  I am taking a break from writing new posts through the northern hemisphere summer.  Things will not be totally dormant with the blog as I am going to be trying a few things to stimulate reading of earlier posts so we’ll see what happens!

When I begin writing new post again in September the focus area will be Organization Development. A large topic and where we will be looking an alternative way of looking at and understanding the individual in organizations.

Advertisements

Learning and Development – Measurement

20151104_145237It’s pretty much unavoidable to look at the topic of learning and development in organizations without looking at measurement. Unfortunately the topic of measurement in learning and development often creates levels of frustration that no other area of learning and development does.

I think a lot of this frustration, a lot of the OUCH! is created by the typical way in which we understand organizations which of course informs how we understand learning and development. As noted in earlier posts the primary way learning and development occurs in organizations is through content focused events. Even though these are seen as cost effective they are still expensive. The real problem though is that they are typically seen as the only thing, the only activity that is supposed to change behavior and thus positively affect performance in the organization.

If you have a single event that has a large price tag and that single event is supposed to be the primary variable in affecting performance it makes all kinds of sense to ask, ‘what is the return on this investment?’

I think that in many ways the frustrations felt in trying to respond to this question are not so much frustrations with the question itself, but that the question surfaces the real problem of content focused learning events.

They don’t change behavior!

We all know this but we continue to engage in these singular events and then end up doing a whole lot more non valuable work trying to measure their impact and it cannot be effectively done!

OUCH!

Given that the point of L&D initiatives is to change behavior you have a real problem when the above question is asked if your primary design for learning is a content focused event.

A lot of the OUCH! in measuring the effectiveness of learning and development disappears when the design shifts to extended time frame context focused initiatives. When we look at things like executive coaching, mentoring and even action learning initiatives two things tend to happen in terms of measurement:

  1. It is not a priority
  2. It takes a subjective or qualitative format

It could be argued that this happens because this type of learning design tends to be reserved for more senior people and they have the power to legitimize these two points. You could also argue that the effectiveness of the design itself is what is causing the above to occur. My guess is that it is both. But if you are in an organization or situation when measurement of L&D is a priority the second point is very important.

The most effective way to measure the impact of learning and development is to use subjective or qualitative methods.

If you want to go deeper into the details of qualitative measurement I have found value in the book  Qualitative Research & Evaluation Methods: Integrating Theory and Practice by Michael Quinn Patton. There are other resources focusing on this area as well if you check into it a little further.

In a nutshell however, especially with L&D initiatives qualitative measurement is most effective when it takes the following format:

  • Collection of individual ‘stories’ of application and impact of the learning initiative to business scenarios.
  • Analysis of enough stories to extract ‘themes’ of the impact.
  • Sharing of these themes and making the actual stories available for review by others.

In many ways the informal process of sharing stories is what sustains the use of things like coaching and mentoring. You will often hear people (often senior people) who passionately tell their stories of how valuable a coaching process has been or how much impact a mentor had on them early in their career.

As you move to extended time frame, context focused designs you are really just formalizing and doing a bit more analysis of this story sharing process.

This type of measurement or evaluation is a shift from the typical attempts at quantitative evaluation so it is important to incorporate this shift right into the design of any initiative. Trying to add this on to the end of an initiative typically is quite difficult. People need to know how evaluation is going to happen so they are prepared and can consider their stories right from the start.

The other thing this type of evaluation does is put the primary accountability for evaluation on the learner and how they are applying their learning in a business context. Most quantitative evaluation methods of learning initiatives do a very poor job at this.

If you are in an organization that is adamant about measuring the return on investment of learning, the faster you can get to qualitative evaluation the better. The causal factors affecting the value and impact of learning are very complex. Quantitative evaluation of learning almost always will force you into looking for simple causal (often one to one or A to B) factors. Since these do not exist for complex learning topics your quantitative measures are always at risk of scrutiny by someone who wants to question them and you will be mostly defenseless when this happens.

Moving to a qualitative evaluation process inhibits this significantly. It is very hard to refute a large number of practical stories from participants that say the learning is having a business impact. On the other hand it is also very hard to refute a large number of stories that say the learning is having little impact!

However isn’t this what we want from our evaluation of learning?

What are your learning evaluation stories?

 

 

Learning Content Without Certainty

20151104_145237When I was thinking for a title for this post and came up with the four words above I had a kind of twinge as I typed them.  As I thought about that twinge I realized that it was coming from my own drive for certainty; my own left loop in that regard. After all, don’t we all want to be certain that if we are going to invest in an initiative to learn something that the result will be positive and of value?

It may be this twinge, however it may manifest itself, that draws us to the type of learning content discussed in the last post. Learning content that in some way or other ensures that by using that content effectively will produce some kind of a result.

Unfortunately, given the complexity of human interaction, founded on transformative causality, no such guarantee is possible. There simply are not, 4 levers, pulled effectively that will give you the culture you want for example.

So what are we to do? Short of scrapping most existing learning content I think there are three areas that can be good starting points:

  1. From existing learning content remove:
    • The ‘proper’ way to apply that content.
    • The connection to the production of some desired result (certainty).
  2. Focus on content that helps us think differently about our own interaction models.
  3. Legitimize that any learning content requires an extended time frame and context focus.

Point 1 above is interesting when applied.  The first thing that happens when you remove the proper way to apply the content and the connection to certainty is that you are left with concepts. The second thing that happens is that a lot of the actual learning content disappears.

For example currently there is a ton of learning content on the need for effective conversations. Effective conversation is a concept and there is no doubt we need to be having effective conversations in our organizations.

The bulk of actual learning content however is focused on the ‘proper’ way of applying that concept and given this proper application the assumption is that you will get what you want.

In order for this content to exist it must be generalized and as unfortunate as it may be, none of us exist in a generalized world. We exist in worlds where having an effective conversation means dealing with a very specific person in a very specific context. There is simply no ‘proper’ way to approach this scenario and there is certainly no guarantee of success.

Before we had this onslaught of learning content about effective conversations we probably would have done one of the following if there was a need for an effective conversation:

  • Nothing
  • Tried something and see how it went and moved forward from there
  • Asked someone about how to best approach the situation and moved forward from there

And this brings us to point 2 above – focus on content that helps us think differently about our own interaction models. Trying something or asking someone are quite simple examples of this and in essence is the foundation of the learning design discussed in earlier posts.

Interaction Model

Since our patterns of interaction and behavior tend to stabilize over time we may need help in recognizing our patterns and altering them and this is where learning content may be valuable.  This content will help us think differently. It will not prescribe how to ‘do’ differently. That part is always up to us and is specific to the environments we find ourselves in.

Self ManagerWhat Dr. Freeberg described as the Self Management Model is an example of a thinking tool (learning content) that I positioned in the series of posts on strategy. The concept being focused on is self management and this content provides a way for us to think about our current interactions with our manager, our patterns of interaction and experience and as well our intentions for future interactions with that manager.

You probably have other learning content such as this that helps you think differently about your own interaction model. One very common aspect of this type of learning content is that you can use it again and again depending on the situation.

This brings us to point 3 above – legitimize that any learning content requires an extended time frame and context focus. It is the legitimization that tends to be the challenge here; especially when learning initiatives are focused on larger numbers of people. The posts on learning design focused on legitimizing this point.

What the above 3 points bring to learning content is much more of a match between the content itself and our actual experiences of being in an organization. This reduces OUCH! and this case reduces OUCH! in learning and development. What this match between learning content and experience exposes is the very real and difficult challenges we have with learning in organizations; learning in general. It is hard to alter our left loops; it is hard to bring new intentions to interactions that may have been stable for years.

It is learning content that exposes this difficulty that is most effective in our organizations. It helps us think about our interactions seriously and perhaps change them. And it is the change in our interactions that changes us and our organizations.

What learning content (concepts) have you found most valuable?

What Are We Learning and Developing?

20151104_145237

The posts in this Learning and Development series so far have mostly focused on the process of learning. This post will look at the actual content of what is happening in that process; what is being learned.

You get a stable left loop in the interaction model through the process of numerous interactions.

Interaction Model

Interactions that over time establish a pattern that is similar in both content and process. In the post OUCH! What’s the Purpose of This? one of the points why I am doing this was noted as:

  • To illustrate that most organization theory and thus formal practice supports a drive for certainty as well as seeing the individual as a discrete and separate entity distinct from the contexts they experience; and that this theory does not match our experience.

In order for these things to exist as stable left loops it has to mean an awful lot of what we are learning and developing is this drive for certainty and that the individual is a discrete and separate entity distinct from the contexts they experience.

This type of learning content feeds the OUCH! in our organizations constantly. And it seems that over time, as this left loop has become more and more stable, we have an almost insatiable appetite for this learning content, perhaps better described as an addiction!

So when it comes right down to it the area of Learning and Development is really mired in a lot of OUCH! producing patterns! Not only does the process of how we typically understand L&D produce OUCH! so does the content we feed into that process! And of course that content then informs and further stabilizes our left loops in things like performance management and strategy.

As a brief example:

About 5 years ago I was on Twitter reading some tweets and came across one by a very well recognized organization guru. The tweet basically read that if you ‘pulled these 4 levers’ you would get the culture you wanted in your organization.

Normally after reading such a thing I would just go bang my head against the wall a few times and leave it alone but this time I actually tweeted back – So if we pull these levers we should get the culture we want? And if we don’t get it are we incompetent? I did not expect any kind of response but I actually got something back! The tweet said they were not sure about incompetent but you should get the culture you want if you pull those 4 levers.

So if not incompetent, what is it, if you don’t get the culture you want?!? Whatever you call it, there is a lot of OUCH! just waiting to happen.

This example illustrates one of the most common things to be avoided in learning content:

  • A step by step process that ensures you will get some result.

This type of learning content represents a drive for certainty and more specifically the assumption that power can create certainty.  That power may be positional power but more often with this type of learning content the power is subtly defined as the ‘correct’ execution of whatever steps are listed that will get you some result.

Any learning content that in some way or other ensures that by using that content effectively will produce some kind of a result is problematic.

It is important to be very diligent in looking for this connection in learning content to a drive for certainty.  Not only is it often subtle but our own stable left loop in assuming certainty can be produced by power can really constrain us in seeing the subtle links.

Given this, and although this may seem a little drastic, I think we are best to be highly skeptical of ANY learning content since so much current learning content is in some way connected to this drive for certainty.

Learning content that has a connection to a drive for certainty will typically have two common elements:

  1. There is a ‘correct’ way to apply this content.
  2. That correct application will produce a (desired) result.

When the desired result is not achieved these two things then create:

Blame, guilt or shame either directed at the learner (shame or guilt), the situation/people the content is applied to (blame) or the content itself (blame).

We can help ourselves be diligent and skeptical of learning content by asking ourselves if it has the two elements or the outcome listed above.

Learning content that positions and develops the individual as a discrete and separate entity distinct from the contexts they experience is in many ways a special version of the drive for certainty. I will be looking at this in a more focused way in the post series on Organization Development so will put it aside until then.

For now I would encourage us to take a very critical look at the learning content we have in our organizations through the lens of what is written above. It would be interesting to see some discussion on actual learning content and some debate on whether or not it is connected to the drive for certainty.

Comments, thoughts, examples, debate?

The next post will focus on learning content that stays away from this drive for certainty.

 

L&D – A Real Story

20151104_145237

Right in the middle of writing these posts on extended time frame and context focused learning designs I received a phone call from a potential client. It almost seemed like a test. A test to see if what I was writing about was just that, writing about something but perhaps not about actually doing what I was writing about.

The request was this:

We have a group of around 50 people and we want to use one of your assessments (the Team Management Profile). The leader of the group is getting these people together for a day long meeting and has allocated 90 minutes for your segment. Can you do this for us?

As you can no doubt tell I was being asked if I could run a content focused event and make that event magical so it would add value for this group. Pretty much what I have been writing about as being a waste of time in terms of behavior change.

The person I was talking to was a senior HR person and was skeptical (a good sign!) and asked right up front if I thought this was a waste of time (and money). The interesting thing about requests like this is that they are a really, really good opportunity to play all the roles that have been discussed in this series of posts.

The reason it is a good opportunity is that the ‘event’ is already so short time wise that everyone knows its value is questionable if that is all you are doing! If the event is longer, even half a day, people seem to think magic is more likely to happen.

So right off the bat you have a chance to play the Designer and Educator role.  Below is a summary of our interaction.

I agreed with the senior HR person that if all they were going to do was the hour and a half, it would be better not to do it as the value would be minimal. I said however that the hour and a half could have real value if we designed in some before and after interactions.

We discussed and then I sent her a summary of what that might look like:

  • Everyone would receive the results of their assessment prior to the event and would be required to review it and as well watch two short videos that provided some background to the assessment.
  • After the event (which would focus on a very specific part of the assessment) I would follow-up with the group one week later with an email explaining how to use a specific part of the supplemental material that comes with the assessment and ask that they use it and review their experience of using it at their next regular team meeting.
  • A month after that another email would go to the group explaining a second part of the supplemental material and how that could be used by the team.
  • The overall leader and leaders of the sub groups of this larger group would be responsible for ensuring this follow-up use of supplemental material happened and I would communicate with them via email regarding what this support could look like.

I had put time estimates to each of these areas and my contact went to the leader of the group and subsequently the decision to move forward was made. This was quite an expensive initiative and I think the decision to move forward made much more sense when the extended design was discussed. It became quite obvious that the potential for value was far greater with this design.

As it happened, the organization decided to accredit two internal people to use this particular assessment internally and it will now be those two people who drive the sustainable use rather than me. They already have different ideas regarding how this will happen that are more specific to their understanding of the organization and I am quite sure they will be more effective than I could be with their capacity to be much more flexible and available than I would be.

When you think of the roles that have been discussed in these posts you can see each of them playing out here.

Role Mantra

You can also see them playing out in a very typical situation in the L&D arena in ways that recognize the stability of the left loop regarding learning in organizations and how it is imperative to try to change things incrementally if needed and dramatically when the opportunity presents itself.

Perhaps the most important design impact this type of interaction has is that the design is focused on the self managed learner; the most important people in our organizations.

L and D 2

The event and my role as expert are not the key variables in behavior change. The design provides self managed opportunities for people to sustain the use and value of the assessment. It is evident to all that the participants in this initiative have the accountability (and opportunity) to sustain its value.  I am certain there are a lot of self managed learners in this group that will do just that.

This also freed up the event to be just what it could be; an opportunity for people to interact a lot about a specific part of the assessment data. We didn’t have to worry about takeaways, what was learned or next steps. I would imagine in a few weeks time people will not even remember who the facilitator was, but they will still be using the content and the concepts available to them.

It seemed so right to get this call right in the middle of these posts (remember that synchronicity thing from the strategy post!) Shifting our thinking about L&D and creating different designs that have much less OUCH! can fit into our current scenarios and situations. It does not have to be incredibly hard, different or new. It’s really about creating our own new left loop, one interaction at a time….

A Design Framework for Learning – 2

20151104_145237This post will take a look at some variations in the extended time frame context focused learning framework design illustrated in the last post. It is important to recognize this design as a framework rather than an ‘answer’ and that underlying the framework are the four primary design elements affecting behavior change:

  1. Extended time frame for numerous interactions
  2. Performance pressure
  3. Concepts to experiment with to change interactions
  4. A process for learner reflection

These design elements can be incorporated in countless ways so let’s investigate some of these as variations in the design looked at in the last post. When looking at variations the text in red is really most important in terms of what you are trying to accomplish. Underneath the red text are listed some other options.

  1. A small internal learning ‘steering committee’ is formed to oversee the initiative. This group is composed of volunteers that are directly involved with the learning initiative. The purpose of this is to drive internal capacity plus to continually promote the value of this design to offset the power of existing ways in which learning and development are understood.
    • The ‘steering committee’ may just be one person; especially if you are working with a single team. Often the team leader will play this role but I find it even more effective if a team member volunteers. This helps to drive internal capacity.
    • Often this roll will rotate to other members of the group over time.
    • If you are working with non intact teams (i.e. a group of managers/leaders) the steering committee may be some internal function (L&D for example) within the organization. When this is the case they have a very important Educator role to play with participants!
  2. The steering committee meets with the external consultant to overview the process and determine the role of the steering committee. Educator and Consultant roles are being played by the external consultant to explain the rationale of the design and to continue developing internal capacity.
    • Whoever the steering committee is the role of Educator and Consultant are primary in terms of interacting with participants. In some organizations this will mean the steering committee needs to have enough power to effectively play these roles.
  3. The steering committee determines the composition of the learning groups that will sustain the learning (most often these are intact teams). Again, driving internal capacity and ownership.
    • The challenge here is often with non intact teams. This challenge is most effectively dealt with by ensuring the context about which the learning group will meet is important to the learning. The context must have business relevance.
    • In these situations learning pairs are often more effective than groups and it is easier to find a relevant context with a pair rather than a group.
    • The Educator role is important here as the design (including learning groups/pairs) and its relevance need to be communicated to participants early and often.
  4. The steering committee communicates to all participants as needed to overview the process and details.
  5. The first ‘learning event’ is held which provides an initial learning concept to all participants and modeling of next steps (typically no longer than 1/2 day or less if virtual learning is utilized). This is the Facilitator role with a bit of Educator and Consultant mixed in. Note that ‘concepts’ are provided as much as possible rather than content (see this post).
    • If you have learning content that has been introduced in the past but has not been effectively applied you can re-introduce it in this design.
    • You may not need a learning event at all if you already have learning concepts out there (this can include things like values, diversity initiatives, innovation, even culture change). You are then just introducing the idea of the learning group and in essence, step 6 below.
    • If this is the case do not eliminate the need for a steering committee and treat the initiative as a new initiative.
    • Virtual learning events fit well here.
  6. Learning groups meet once a week (minimum) to apply the learning concept to a real business issue for a period of 4 – 6 weeks. The learning is therefore focused on business context. Normally these meetings are regularly scheduled intact team meetings so no significant disruption to the business is required.  See below for a critical component to this design element.
    • This process can work very well with virtual teams in their normal virtual interactions. The Fly on the Wall process works well.
    • With non intact groups the meeting schedule and context relevance  is best managed by the group or pair. Making this visible is part of the Educator role of the designer and steering committee.
  7. The steering committee monitors the process and interacts with the external consultant if changes to the process need to be made. Monitoring may take many forms including measurement of impact. We will look at this in a future post.
    • The most important thing to monitor is if the learning groups are actually meeting. If the context is important to participants, learning meetings happen.
    • The second most important thing to monitor is the impact of the learning concept on business issues addressed. Real stories of impact work best.
    • Sharing of these stories with the larger group is typically important.
  8. Steps 5 – 7 are usually repeated two more times with different concepts being provided for each learning event. This increases the potential for behavior change plus establishes a new pattern of interaction for applying learning which is building internal capacity.
    • Don’t be surprised if your planned learning concepts need to change.
    • You may find learning groups want to change the Fly on the Wall process as time progresses. This is typically a good sign but remind groups to make sure they don’t lose the focus on learning concepts or the need to disrupt stable patterns of interaction at some points in their work.

As you likely have noticed what is being created is a new left loop in terms of understanding and acting on how learning and development occurs in organizations.  In essence what this design framework is doing is legitimizing how learning normally occurs in organizations; one interaction after another with new concepts being introduced into those interactions.

The OUCH! of content focused events is reduced and the challenge and sometimes pain of actually changing behavior is front and center.

The points above are just some variations and considerations that can surface with this design framework. You may have other ideas to offer or questions to ask. If so add your comments and let’s see what emerges!