Tag Archives: Giving and Receiving Feedback

On-Line Workshops are Better Than Face to Face

I decided several years ago to launch on-line research-grant workshops purely as a convenience measure. Over the last few months I have created the materials for a fully on-line workshop. As the on-line workshops launch, I am a little bit surprised to realise that they are better in almost every respect than the face-to-face workshops that I have been delivering for the last five years.

I suspect that on-screen I am less engaging than in real life but I know that in every other respect the on-line workshops are better. The on-line lectures are shorter and clearer. They are supported by well-structured written material. The on-line workshops offer opportunities to get feedback. And they give participants more flexibility and more time.

Better Lectures On-Line

My face-to-face workshops were almost entirely lecture-based, and always received rave reviews. So naturally I assumed that recording a face-to-face workshop would produce excellent lectures. The recorded lectures were fine except for two problems. The picture quality was appalling. And the audio content was dull, repetitive, and full of speech tics and idiosyncrasies. Clearly I had to take a different approach.

The cancellation of all my face-to-face workshops at the end of January created the opportunity I needed. I set up a studio at home and scripted and recorded new lectures. I enjoyed the recording and editing and I am very pleased with the results. The video lectures are clear, crisp and to the point, while retaining enough editing imperfections to create an impression of authenticity. Friends assure me that the appeal definitely comes from the quality of the content rather than the slickness of the production.

Written Material

The lectures are supported by extensive written material, which was originally intended to be published as a book, and may yet be. This has allowed me to resolve a long-standing problem with the face-to-face workshops. Although they were supplemented by slides, handouts, and blog-posts, the material was fairly disorganised. Now each lecture sits on a web-page. The web-page contains text that develops the points made in the lecture. The web pages are organised into three strands that address the main needs of workshop participants.

  1. They need to understand strategy: how to plan grant writing, what to do before starting to write, and what to do after finishing.
  2. They need to understand tactics: the characteristics of a good grant application and what to do to produce one.
  3. They need to develop skill: the ability to write the kind of text needed in a good grant application.

Feedback, flexibility and time

It has always been difficult to work on skill in face-to-face workshops. Skill only comes from practice. People get better at writing by practising their writing, and by getting feedback on what they have written. In a face to face workshop most of the time is taken up explaining tactics so there is little opportunity either for participants to write or for me to give feedback. In the on-line workshop, all the material is pre-recorded, so participants can practise writing and I can give feedback on what they write.

The principal cost of a workshop is determined by the amount of time that I have to spend presenting it. In a face-to-face workshop, the presentation time has to be a continuous block, bracketed by travel time to and from the venue. The need to travel meant that short face-to-face workshops were uneconomic, except very close to home.

In the on-line workshops, I stay at home, and all the material can be presented, for as much time as the client wants, without me. So my involvement can be as little or as much as the client wants and can be recorded, so that participants can choose when they want to engage with the material and with me.

The Fly in the Ointment

One surprise was that I expected that switching to on-line delivery would reduce stress but initially stress increased. I have decades of experience with with face-to-face delivery. I know how it works, I can see when something goes wrong, and I know how to fix it. On-line workshops depend on web-page components that I don’t fully understand and it made me anxious that I definitely can’t deliver a workshop if they fail. However, after more than a year of trouble-free delivery I now have more-or-less the same relationship with on-line delivery as I have with air-travel. I don’t understand the machinery that makes it work, but I do now trust it to work.

Ten questions to Get Feedback on a Grant Application

FeedbackThis post was triggered by an  article in Harvard Business Review about how to get feedback on your performance at work. Getting feedback that you can trust is a universal problem and it affects grant-writers more than most. The key to getting good feedback  on a grant application is to think carefully about whom you should ask and how you should ask them.

Whom should you ask for feedback on your grant application?

The best option is to ask the right kind of expert, if you can. Expertise on grants is much more important than expertise in your subject. The ideal person would be someone who has served several years on the grants committee that will consider your proposal. Failing that, you could ask someone who has wide experience of similar grants committees and a good understanding of the scheme for which you are applying.  Unfortunately these people tend to be busy and you may be in a hurry. You can ask Parker Derrington Ltd to read your grant application and give you face-to-face feedback by Zoom, but you would have to pay. What can you do if you don’t have an expert on call and can’t afford to pay one?

If you can’t get an expert, you should choose an ignoramus, or someone who can act the part as far as your project is concerned. You need someone who does not share your views about what is important in your subject but who is open-minded. This mimics the kind of expertise that a grants committee member will have. The ideal person would be someone at your level, in a different but related field. It can be a good idea to exchange feedback with a colleague in a different subject area who is also preparing a grant application. It is a mistake to use someone who knows as much or more than you do about the area of your project, unless they are sufficiently experienced in the way grants committees work that they will be able to discount their knowledge of your subject.

How should you ask for feedback on your grant application?

If you ask a real expert for feedback, it doesn’t matter how you ask, they will give you useful feedback. Otherwise, you need to design your questions carefully if the answers are to be useful. Here are some suggestions.

  1. Write down the main outcome of the project. Mark where it describes it in the text with a (1)
  2. Mark with (2) in the text the section that gives evidence that the main outcome is important. Can you think of anything else that could be said to support the claim that the outcome is important?
  3. Is the importance of the outcome specific to this funder? Mark any references to the funder’s goals with (3).
  4. How many separate components does the research project have?
  5. Mark with (5-1), (5-2) etc, the sections of text that state the outcome of each component of the research project.
  6. Mark with (6-1), (6-2) etc, each section of text that explains how important the outcome is of a component of the research project.
  7. Mark with (7-1), (7-2) etc, the sections of text that give citations that support the explanations in (6-1), (6-2) and so on.
  8. Mark with (8-1), (8-2) etc, the sections of text that describe the research processes that will produce the outcomes (5-1), (5-2) etc.
  9. For each of these sections, can you mark with (?) any processes that are not clear, and can you suggest any extra processes that might be needed in order to produce the research outcome.
  10. Mark in the text with (1) where it states what will be done to make sure that the overall outcome of the project has maximum value to the funder. Can you think of any way that this could be made more valuable to the funder.

You should ask the reader to spend no more than half an hour answering these questions. If they are unable to answer them in that time then you need to do some serious rewriting.

Finally, you should know that the correct answer to question 4 should be three.

Put some meat in your feedback sandwich

7213133_s

Make sure that your feedback sandwich has plenty of meat.

This is one of a series of posts with advice for people who review grant applications for their friends and colleagues. It is also intended to encourage people writing grant applications to review what you have written before you ask a colleague to read it and tell you something you could have worked out for yourself. My last post argued that a quick and easy way of reviewing a grant application is to check its structure. Good structure makes a grant application easy to read.

Most applications will have very poor structure. If the structure is poor, it is not worth reviewing the detailed content. One reason is that it will take too long. Another is that, even if the content is very good, a funding committee’s enthusiasm for the application will be limited because the poor structure makes it difficult to understand.

At this point I feel honour-bound to confess that I have hesitated a couple of weeks before finishing this post because I know that many people find my feedback a bit too direct. They like to soften criticism by constructing a feedback sandwich that smothers useful comments in copious quantities of praise. My view is that feedback should be clear, it should explain how the proposal can be improved, and it should be delivered as quickly as possible. Feel free to cut and paste from what follows and to wrap it up in whatever form suits you.

The application should have an introduction that states clearly:-

  • what goal the project will achieve;
  • why the goal is important;
  • about three aims, which are things that we need to discover or understand in order to achieve the goal;
  • the general research approach;
  • about three research objectives, which are pieces of research that will enable us to achieve the aims; and
  • what will be done with the results.

These statements are what I call the ‘Key Sentences’. They are a distillation of the argument that the project deserves funding. If the grants committee believe the key sentences, they will almost certainly award a grant. So the rest of the case for support should be designed to make the reader believe the key sentences. It does this by repeating them, and reinforcing them with evidence or explanation.

After the introduction, the case for support has two main sections, each of which repeats six of the key sentences and then justifies or explains them. The background section deals with the goal, the importance and the aims. It should explain, with evidence, why the goal is important and how meeting the aims will achieve the goal. The description of the research project deals with all the rest. It should explain in detail the research approach, the research objectives, and what will be done with the results.

The structure allows a reader to understand the message of the grant in 2 or 3 minutes, by ‘speed-reading’ it. This is very important. The committee that decides whether or not to award the grant will have dozens of other applications to read and discuss. Most members will have a limited understanding of the detailed argument. Some of them will still be reading the application during the final discussion about whether to fund it. All of them will have a say in whether it gets funded. They will be much more enthusiastic to fund an application that they can understand and remember.

The structure also helps the referees who will read the proposal thoroughly and write an evaluation. Each key sentence leads directly to the detailed argument and evidence that justifies and explains it. It is not quite as important to make referees’ task easy because each referee typically only deals with a small number of proposals. Moreover, their enthusiasm will be based on their evaluation of the detailed argument. But it definitely doesn’t hurt to make their task easy.

Opening sentence

The opening sentence must whet the reader’s appetite. The best way to do that is to say what the project will achieve, together with something about how it will achieve it, ideally with a clear implication that the research team are well qualified to do the work. For example, the sentence “This project will develop a new potential treatment for stroke based on a family of synthetic metabolic inhibitors that we have discovered, and synthesised” achieves all this. The reference to discovery and synthesis suggests that the project is a continuation of past work. Of course the suggestion that the project continues previous work does not testify to its importance – that will rest on the opening clause, which states that the project will develop a new potential treatment for stroke.

Draft grant applications quite often have a good opening sentence. But it is not usually at the beginning. Usually it is somewhere towards the middle. Skim through the whole case for support to see if there is a suitable opening sentence that can be moved to the opening position.

It is useful to repeat the opening sentence at the start of the background section, which usually follows after the introduction. It is also useful to open the summary with the same sentence. Repetition of this kind is very good: it helps the reader to remember the essential message. Unfortunately most academics don’t like to repeat a message unless they can confuse readers by using completely different words.

Importance sentence

It’s useful to have a sentence explaining that what the project will achieve is important and to have it follow the opening sentence, which says what the project will achieve. Health, social, or economic benefits are usually, depending on the remit of the funder, good reasons that something is important. Solving a major theoretical problem may also be a good reason. Being the next logical step from the applicant’s previous research is typically not a reason for importance, although mentioning this kind of progression can be a useful way of supporting the proposition that the applicant has the necessary skills to carry out the research.

The importance sentence should also be repeated in the background section and in the summary.

Aims

The aims of the project should be stated immediately after the importance sentence. If you don’t know the difference between aims and objectives (surprisingly, the Oxford Dictionary is rather little help: its definition of ‘objective’  is ‘thing aimed at’), read this excellent post on Pat Thomson’s blog.

I suggest that there should be about four aims. I also think that because funding agencies often ask you to state aims and objectives, you should use them as key parts of the structure. A good way to state an aim is to say that “We need to know” something. There are three important things to check about each stated aim.

The aim should be restated, word for word, in the background section, where it should be followed by a few paragraphs that convince the reader that this is an important aim. Usually they do this by citing appropriate literature. Stating and justifying the aims like this is a device to create a direct link between what the project will do, and an important underlying question that the funder will pay to have answered. It’s also a way of papering over the cracks when the link is a bit tenuous.

The aims of the project should also be stated in the summary. Repeating them exactly, using the same sentences, is a good way of helping the reader to remember them.

Project overview sentence

It is useful to have a sentence that says what kind of research project is envisaged and what the likely outcome will be. It’s often necessary to have something that puts the reader in the right frame of mind for the subproject overview sentences. This sentence should be in the introduction and should be repeated at the start of the section of the proposal that describes the research project in detail.

Sub-project overview sentences (Objectives)

The essential difference between aims and objectives is that, aims tend to be abstract and objectives tend to be concrete. Aims are things that we would like to achieve. Objectives are the concrete things that we will do in order to achieve those aims. Thus the four or so sub-projects that comprise a research project are objectives. Matching the objectives to the aims is a good way of convincing the reader that the research needs to be done: the description of each sub-project should begin with a sentence that includes the phrase “this will tell us” whatever the corresponding aim says “we need to know”.

These sentences should be ‘pre used’ in the introduction and in the summary.

Dissemination

The last substantive sentence in the introduction should say something about what will be done with the results. Again, this sentence should be re-used to introduce the last sub-section of the description of the research project, which should explain it in detail.

So, that’s the filling for the feedback sandwich. How do you wrap it up? I think that it’s up to you to use your judgement. All the wrapping does is to set a baseline. In my view it doesn’t much matter where the baseline is, as long as it is absolutely clear how much the structure of the case for support could be improved.

I should point out that some funders, including several UK Research Councils, require applications to contain a section detailing the achievements of the research team.  That section, which is an excellent way of supporting the competence proposition, is in addition to those discussed in this post.

Review a Research Grant Application in Five Minutes

PeerReviewCartoon

Cartoon by Nick D Kim, http://strange-matter.net

This post tells you how to do a five-minute review of a research grant application. If you are asked to comment on a grant application by a friend or colleague, you should begin with this five minute review. In 95% of cases it will be all you need to do. Except create the feedback sandwich of course.

If you are writing a grant application, before you ask anybody else to read it you should spend five minutes reviewing it yourself. Far too many of the grant applications that I get asked to read take me a lot less than five minutes to review. Then it takes several days to construct a palatable feedback sandwich with the filling  “rewrite this completely”.

I don’t say you can review the detailed content of a grant application in five minutes. That takes longer and I will write about how to do it quickly in a future post. However, five minutes is plenty of time to review the framework of the case for support and check that it is appropriately used.

The framework is important for two reasons. First, if it is good, it tells the reader the essential story of your grant application very quickly. Remember, most readers only want to know the essential story. Second, the framework guides the reader to the detailed content that supports and justifies the essential story, so that the detail can be reviewed effectively and quickly.

‘Summary Sentence’

The most important part of the framework is a summary sentence. This should say what the project will achieve and enough about how it will achieve it to give a bit of distinctiveness and a bit of plausibility. It is essentially the elevator pitch, except that instead of taking 30 seconds to 2 minutes to say it or read it, it should take more like 15 seconds. The summary sentence should be the first sentence of the introduction, so it should take no time to find it, you still have 4 minutes 45 seconds left.

If you read my last post you will recognise that the summary sentence is what I refer to there as the first key sentence. So it will come as no surprise that you should check that the summary sentence is re-used as the first sentence of the part of the case for support that sets out the background to the research project. Let’s allow 15 seconds to do that. 4 minutes 30 seconds left.

‘Importance Sentence’

Now you should spend 20 seconds checking the second sentence of the introduction. It’s the importance sentence. It should say something about why it is important to achieve whatever the project will achieve. Make a mental note about whether the importance sentence has a practical element. Does it mention a real world problem – childhood cancer, the economy, forgetfulness in old-age, or some such. It’s not essential that there is a practical element to the importance sentence but it has implications for the dissemination sentence, which you will review in a few minutes time.

As soon as you have checked the importance sentence for practical content, jump to the background section and check that the importance sentence is repeated there. This is just a quick glance, so it should only take you 10 seconds. You have 4 minutes left.

‘Aims Sentences’

Go back to the introduction. Immediately after the importance sentence there should be a set of aims sentences, about four sentences setting out the aims of the project. The aims sentences should state things that we need to know, understand, characterise or in some way discover. The aims sentences could be preceded by a linking or framing sentence and they could easily be formatted as bullet points. Checking that the aims sentences are in the right place should take you 10 seconds. Do not look at the content of the aims sentences yet. Wait 30 seconds until we get to the sub-project overview sentences.

Now you should check the next sentence of the introduction. It should be the ‘project overview sentence’.

Project overview Sentence.

The project overview sentence gives a simple one-sentence overview of the project. It might also be structured as a linking sentence to the four or so sub-project overview sentences, which should immediately follow it and which could appear as bullet points and could be expressed as objectives. It should take you 30 seconds to read the project overview sentence and 20 more to check that it appears again to introduce the part of the case for support that gives the detailed description of the research methods and of the project.

Next you can spend 10 seconds checking that the sub-project overview sentences follow the project overview sentence. Now you are ready to check their content against the content of the corresponding aims sentences.

Sub-project overview sentences; content of aims sentences

Each aims sentence should link logically to the corresponding sub-project overview sentence in the following way. The aims sentence should say “We need to know X”. The sub-project overview sentence should give a very brief description of the research in the sub-project and should end with a clause that says “this will tell us X“.

The aims sentences should be repeated in the background. Each of them should introduce a discussion of the evidence that supports the assertion that this is an important aim.

The sub-project overview sentences should be repeated in the part of the case for support that describes the project in detail. Each of them should introduce the detailed description of the corresponding sub-project.

This checking and matching will probably take nearly a minute to do for the first aim/sub-project overview sentence pair but the others should be much quicker. It’s just a matching exercise after all. Lets assume it takes 2 minutes. You have a minute left, and only the dissemination sentence to check.

The dissemination sentence

The last sentence of the introduction should say what will be done with the results. It is the dissemination sentence. You need to check that it is repeated in the description of the project as the introduction to a section on dissemination. You also need to check that, if the importance sentence has a practical component then the dissemination must have a corresponding practical element. For example, if the project is going to discover a cure for a disease, the dissemination needs to promote the use of the cure in some way. As a general rule there must be a plan to disseminate the results in a way that makes it likely that the claimed practical benefit will be realised.

What to do with the outcome of the review

Only about 5% of grant applications will pass this five minute review. If you are reviewing a grant application for someone else and it passes, it should be pretty easy to review the detailed content. I’ll give some helpful pointers in a future post. It goes without saying that if the grant you are reviewing is yours, it is safe to give it to someone to review.

In the more likely case that the grant application fails the review, then it needs rewriting. If it’s yours, that’s pretty simple. Read my post from last week. Write the 10 key sentences and use them to organise the text you have already written. If you can’t write the 10 key sentences then you shouldn’t be writing a research grant.

If the grant application is not yours, you need to write a feedback sandwich. Basically the grant needs to be rewritten before it’s worth looking at the detailed content. I’ll say something about how to do this in my next post but whatever you do don’t use the phrase “completely rewrite”.