Off and Running
Typically, the competition staff and stewards arrive at the site an hour or so before judging is to begin because a number of tasks need to be completed simultaneously in a short period of time. Stewards are generally responsible for setting breakfast food out if this is being provided, as well as configuring the tables and chairs in the manner the organizer desires.
If the room is already set up with tables and chairs as desired, the stewards can begin the task of preparing them for judging. This includes covering tables with butcher paper or tablecloths if desired, filling pitchers with ice and water (if bottled water is not being provided), cutting bread and putting into bowls, baskets or on plates. Matzo, salt-free crackers, or unsalted oyster crackers can also be used instead of the bread. One of each of these items is put onto each judging tables.
An adequate number of mechanical pencils, a stapler, extra staples, a bottle opener, calculator, dump bucket, and a flashlight (if desired) are also added to each judging table. Stacks of cups are opened, cups removed, and placed on tables for every judge team. Style guidelines, flight summary sheets, scoresheets (beer, mead, or cider, as appropriate for the category), and cover sheets adequate for the number of entries to be judged that session are added to the table as well.
Making the best use of the space and keeping teams judging split categories in close proximity, table tents or signs noting the style or category to be judged are placed on the judging tables. Table tents are simply pieces of paper folded into a triangular pyramid shape so they stand up on the table. Placing them on the appropriate tables helps the judges find their judging station and the stewards know where to deliver entries.
If walk-in entries were allowed, the registrar and an assistant check in those entries as they arrive. This includes affixing pre-assigned neck and cap entry number labels to each bottle in the same manner as was done during the unpacking process. Entries then must be sorted and moved to where the stewards will be separating beers into flights or into the cooler as needed. It is helpful to have one individual assigned to this task so that the operations of the registrar and stewards are not interrupted and entries do not get overlooked.
While all of this is occurring in the judging room, the head steward or designee and an assistant pull the cases of categories to be judged during the first session from the cooler and transport them to the judging room. With the assistance of other stewards, the entries for each category are checked against the pull lists, and any missing entries are appropriately marked. If any entries are not found in the appropriately labeled case boxes, stewards should check the stash of walk-in entries that arrived that morning prior to checking with the registrar or judge coordinator on the status of that entry.
Once a flight has been pulled and all entries are in place, the flight sheet can be placed with the category bottles and the whole flight set aside until it is time to begin judging. The crew should keep working on organizing flights until all of the morning flights have been filled. Bottles kept in case boxes stay colder longer, so it might be advisable to keep six-pack carriers in the cases until needed for judging, unless the style such as bitters, meads, strong beers, or some Belgians benefit from serving a bit warmer. Check with the lead table judge to verify appropriate serving temperature.
As judges arrive, they may be requested to fill out a judge registration form or check in with the judge director before the competition begins. Once all judges are in place and vacancies filled as needed, the organizer or judge director opens the competition with some brief announcements. This may include a welcome to the judges, the agenda for the day, expectations of judges such as the range of scores allowed between judges on a single entry (generally no more than 5 to 7 points apart), the minimum score allowed to give an entry (often 13), the minimum score allowed for an entry to receive a medal (typically 30), and any other special rules/exceptions particular to the competition. Be sure to tell judges about logistical arrangements for the judging location, including where bathrooms are located, when lunch will be served, whether there are any parking restrictions in effect, or if the room needs to be vacated by a certain time.
Do not attempt to micro-manage the judging process at a judging table. BJCP Judges have a Judging Procedures Manual that they follow, and judge teams are typically quite good at resolving minor issues at the table. Be aware, however, that some judges may have questions or ask for rulings during a competition. Be open and available to these judges (which is one reason why a competition organizer should not judge). Use the BJCP Competition Rules, local competition rules, and sound judgment as a guide. If you are unsure of how to respond, consult senior judges in the room for their opinion.
Dealing with No-Show Judges
Judges have the responsibility to notify you or the judge director when they are unable to attend judging. Judges who give no notice and who do not show up are a serious problem. However, there are some strategies to minimize the impact of this behavior.
- Make sure you have good contact information for each judge (mobile phone number for calling or texting, email address) so you can contact them on the day of the competition if they are missing. Collect this information at the time of judge registration and consolidate into a master document (spreadsheet or something similar).
- Communicate with judges to confirm their participation before the event (two or three days in advance). Have a mailing list of your judges. This is often a good time to give them preliminary judge assignments, and it reminds them of your event. Ask them to tell you if they cannot attend or if they must be switched to another judging category.
- Plan more judges than you need for the entries you have. If you plan additional judges, then the judges in reserve can be seated into the empty slots. If all judges arrive, seat those extra judges as third judges at a table of their choice.
- If you know you will have trouble attracting enough judges for your competition, consider limiting (or “capping”) the number of entries to a level you can manage. If you force judges to judge huge flights, they often will not come back to future competitions.
After the event and if you have time, contact any no-show judges to let them know they neglected their duties and caused issues at the competition. They may have legitimate reasons due to emergencies. If it is due to negligence, you may maintain a list of problematic judges for future competitions. If they appear at future competitions and perform well, then consider the problem resolved. If there are recurring issues, you may avoid inviting them to future competitions or may reject their request to judge at your discretion.
Remember that communication is key and that judges also are evaluating you. Judges often avoid returning to poorly run competitions, and that includes ones where judge assignments are chaotic or that flights are excessively large. If you are asking for good communications from your judges, remember to do the same yourself and keep them informed about the competition.
Special Considerations for Disabled Judges
Do not bring disabilities of any judge to the attention of other judges unless required to accomplish the task of judging entries.
Whenever possible, do not move a judge with disabilities from the location where he/she has initially been seated (especially if using special equipment). Rotate other judges to them to complete the judge pair.
A judge who cannot judge the color, clarity, head texture and retention of an alcoholic beverage should not be the lead judge in a pair, nor should that judge be paired with a Non-BJCP judge.
To Calibrate or Not to Calibrate
Some competitions start off with a calibration round using a commercial beer or a homebrewed beer that is not in the competition. The calibration round is basically a practice judging session. The intent is to help judges calibrate their palates with the other judges on their team, allowing them to score more closely during the actual judging session. In many situations, the goal of the calibration round is not realized. Unless each judge team calibrates with the style of beer they will be judging in the competition, the practice is generally considered a waste of time and may even confuse the palate. Therefore, many organizers have discontinued the use of a calibration round. The decision to calibrate or not is the responsibility of the competition organizer.
If judging a calibration beer, preparations for the calibration round should begin around the time the announcements begin. Typically this means pouring at least two ounces of beer from the bottles or pitchers provided into plastic judging glasses. These glasses are carried on trays and set in front of each judge in the room. Once all the judges have a glass of the calibration beer, they can begin judging. This gives the stewards a few minutes to deliver all of the morning flights of beers to the appropriate judge teams, matching the category beer name and/or number to the corresponding table tent on the judging tables.
As the judges finish judging the calibration beer, stewards should move throughout the room collecting completed calibration sheets and turning them in to the designated person who calculates the average score of all the judges, as well as the high and low score, and communicates that to the judges. The information provided by this practice may be interesting, but is of little use to the judges. Instead, the judges should consider their score in relation to the other judges on their own team and adjust judging practices accordingly. Once the calibration round is complete, judging of the competition beers can begin.
When there are as many stewards as there are categories being judged at any given time, stewards can be assigned or allowed to choose a single category of beer to work with during a session. Frequently, however, this is not the case and stewards may have to work with more than one category which may add up to quite a number of judge teams. In either case, the steward is responsible for keeping adequate water in the pitcher (or bottled water on the tables), bread (matzo or crackers) in the bread basket (plate or bowl), clearing away the used cups when the judges are done with them on an ongoing basis, emptying the dump bucket when it starts to get full or when a really objectionable entry is dumped, replenishing the stockpile of scoresheets as needed, and getting answers to any questions that the judges may have. Should the judges want to have the second bottle of an entry brought to the table, the head steward should be notified for resolution. In addition, the judges may also choose to have the stewards fill out the cover sheets, calculate the average score of each entry and write it on the flight summary sheet, and/or staple the paperwork together.
Staff and Steward Responsibilities During Judging
Stewards are responsible for assisting judges during the judging process, and interfacing with the competition staff to preserve anonymity of entries. If judges have questions or requests during judging, handle them as best as possible including checking with the competition staff if unsure of something. One frequent request is to check competition entries against the database to ensure an entry is correct, and that all supplied information has been provided to the judges.
Stewards bring the beer to the table for judging. This can be done one bottle at a time, or multiple bottles can be brought at once (this is a common occurrence when using multiple teams on a category, or if judges think the serving temperature is too cold). Handle the judge requests as best you can. They may also request ice if they believe the entries are too warm.
Stewards should double-check the entry numbers before presenting the beer for judging. Announce the entry and category for the judges. Do not open the bottles unless requested. Most judges prefer to look at the bottles and open them themselves. Remember that beer can contain sediment, so be careful in the handling of the bottles.
Stewards should monitor the table supplies and replenish items as necessary. Remove used cups (check with judges first before touching their cups, as judges often save favorite samples for later). Empty dump buckets as they fill, or if an objectionable-smelling entry is dumped. Do not remove bottle caps without checking with judges; they may be needed to identify entries.
Judges may request assistance with paperwork, usually the Flight Summary Sheet and the Cover Sheets. Stewards who do this job for judges will be helping to speed up the competition, and deserve the thanks of judges. Regardless of other requests, stewards should check the math on scoresheets, and scan them for completeness and inappropriate comments.
In general while judging is active, defer to the lead judge at the table unless you are directed against specific competition instructions. Structure your activities so that you are doing your work while the judges are judging; this keeps things moving along. Stewards who anticipate judge needs and who pay attention to the judging process are quite valuable to competition operations.
If stewards are invited to taste while judges are judging, stewards should refrain from commenting until the judges have completed their scoresheets and turned them in, unless directly asked by a judge for an opinion. If a steward has entered the competition, the steward should not work in categories where they have entries.
Judging Split Categories
When larger categories are split and judged by more than one judge team, a mini best-of-show (mini-BOS) round must occur. In these situations, judges from each team judging that category may have the steward recap and set aside up to three entries that they would like to move forward to the mini-BOS. If a flight has no entries of sufficient quality to win (say, scoring under 30), fewer beers can be passed to the mini-BOS. In no event should a beer ineligible to win be passed on.
It is critical for the steward to keep track of those bottles, preventing them from getting dumped with the other bottles that do not move on. For a three bottle competition, the second bottle of the entries would be used for this purpose. There is a box on the cover sheet that indicates that a beer went on the mini-BOS. This box should be checked for all entries moved forward to this round. Especially in big competitions, this is valuable feedback to the entrants. The flight summary sheet also has a similar checkbox; this is helpful for stewards to quickly identify advancing entries.
No additional paperwork is needed for a mini-BOS. A panel of two to three judges selected from the judges that completed the initial judging of that particular category gets together to reevaluate the entries passed forward from each judge team. It is not necessary to select one judge from each flight; the highest-ranking two or three judges from the competition category should be picked if possible. The three beers that best exemplify the style are selected and awarded first, second, and third places. These awards are noted on the cover sheets for those three entries. The entry number of each is then listed in the appropriate place on the flight summary sheet of the senior judge. Depending on the organizer’s philosophy, the final assigned scores for the top three entries can be adjusted up on the cover sheet only to be higher than the scores of all the other entries in that category.
Queued Judging
When a category is split and judges are given fixed flights of approximately equal size, judge teams finish at different times. While 10 to 12 minutes is a pretty standard timeframe for judging a single entry, some judges take less time and others take significantly more. When judge teams finish earlier than the other judge teams in their category, they are forced to wait to begin the mini-BOS. The wait can be mere minutes, but often significantly longer. This can be very frustrating for the judges who finish early and it slows down the competition. A solution to this problem with split categories is queued judging.
The goal of queued judging is for all teams in a category to finish judging at roughly the same time rather than everyone judging the same number of entries. Queued judging requires one steward per category and a table configuration that allows the steward easy access to each of the judge teams. Briefly, in queued judging, each of the teams in a category gets their first entry at the same time. The first team to complete judging their entry gets a second one from the steward. This process continues with the steward handing out entries to the next available judge team, from top to bottom of the flight sheet, until all entries have been handed out. This will likely mean that judge teams will evaluate a varying number of entries, but they should finish their last entries at approximately the same time. The mini-BOS can immediately commence.
Cleaning Up After Judging
Once judging of a category is completed, stewards should begin the clean-up process for the judging area. This includes dumping all the used glasses into the dump bucket, returning opened bottles to the six-pack carriers or case boxes and removing them and the dump bucket from the table and taking them to dumping station. The table should then be straightened up, bread baskets emptied and placed back at the bread station for refilling later, water pitchers removed and taken to the water station to be refilled. When a steward has cleared their responsible judging area, the steward should pitch in and help other stewards with their areas until all the areas are cleared.
All scoresheets should be stacked with the first, second, and third place entries on the top, placed in the folded table tent or designated envelope for that category, and given to the registrar. Alternatively, the scoresheets could be clipped together; however, they should not be held loose. Stewards should check the sheets for accuracy before turning them in, paying close attention to addition, places awarded, and final scores adjusted as needed.
The best practice when the winner is known for a category is to have the cellar master get the winning entry from the cellar and save it for the BOS round. Building the box or boxes of BOS beers as the winners are known saves a great deal of time when setting up for BOS judging.
Typically, lunch separates morning and afternoon judging sessions; however, this schedule can be adjusted if three sessions are scheduled in a single day. A meal should be offered between judging sessions, if possible. If there are teams still in the process of judging when lunch is served, those stewards should remain with their teams until they are finished or can be relieved by a steward who is done eating. About a half hour prior to the scheduled start of the afternoon session, all stewards should return to the judging room to prepare the room and the flights of beers in the same manner as in the morning. Judging also proceeds as in the morning with new flights.
Running the BOS Round
The Best-of-Show (BOS) round determines the overall winner of the competition, and potentially additional competition-specific awards. The BOS round may include beers, meads, ciders, or any combination of the three. Some competitions may not award a BOS (such as if they are a first round qualifier for a larger competition). The BOS round is specifically comprised of the first place winners in each competition medal categories (which may be different than style guideline categories). Some competitions may set further eligibility requirements for category winners to advance to the BOS table, such as a minimum score (often 30) in the main judging round; some competitions choose to limit BOS to beer styles only. However, the fairest approach is for any entry accepted in the competition and winning a category to be eligible to win BOS.
No approach other than judging eligible first place category winners is appropriate for determining BOS. In particular, main round scores should never be used to determine BOS or awards that cross judging flights, since different judging teams might vary in their scoring. Winners should only be chosen through head-to-head judging in a single panel with the same judges. In the event where a category winning entry is not available for BOS, such as a broken or misplaced bottle, or the entrant shipping insufficient bottles, a second place entry should not be advanced to the BOS round. No entry for that category should be represented on the BOS table.
Preparation for BOS judging should be well underway by the time the last session is ending. Entries need to be collected and tables need to be prepared for BOS judging. The BOS round does not need special paperwork, but cups, style guidelines, pencils, bread, dump bucket, and water all need to be in place. If the BOS tables are covered with butcher paper, the judges can write their notes directly on that. Otherwise, blank sheets of paper adequate to hold the number of BOS entries need to be set out for each of the BOS judges. Using pre-printed “placemats” for the BOS round is perhaps the fastest method of all, since the entry information does not have to be transcribed, and stewards can place the beers on the table faster.
When selecting judges for the BOS panel, the organizer or judge director should follow two simple rules. The first is that the judges typically are higher-ranking judges (National or above), when they are available. The second is that a judge cannot have an entry in the BOS, so selection of BOS judges often must wait until right before that round. Note that the most experienced judges, especially those who may have travel some distance to judge, are typically invited as Best-of-Show judges. One of the BOS seats might be given to an honored quest such as a local professional brewer (particularly if the brewery is sponsoring the competition, or is brewing the winning beer).
Generally, the BOS panel consists of three or five judges, depending on the number of entries in the BOS. For specifics, please refer to the Sanctioned Competition Requirements[1]. An even number of judges should not be used since the possibility of a deadlock exists. The benefit of a smaller panel is that the BOS round takes less time to complete as consensus is often easier to reach. A larger panel gets more judges involved and places the responsibility of finding the entry that best fits a style on a larger group of people. BOS should not be used as a “training exercise” for interested but less experienced judges; they can steward and listen instead.
Stewarding the BOS round is different from stewarding earlier rounds. In large competitions, having several stewards works best – one to open and pour the entries and the others to serve them to the judges. Some large competitions use as many as five stewards for the BOS round, one to pull and call out the entries, one to open and pour, and three to serve the entries to the judges. A flight sheet should be created for the BOS round and provided to each of the BOS judges and stewards. Ideally, the flight sheet would include all the additional information necessary for the judges to evaluate those entries with special ingredients. Providing “placemats” printed in the same order as the flight sheets with entry information for each of the BOS entries is even more efficient. Creating these “placemats,” generally six entries to a page, is easy to do when using competition or general office software (such as Microsoft Word or PowerPoint).
BOS stewards follow the order of the flight sheet beginning at the top and working their way to the bottom. It is helpful to put the entries in the same order in the case box as it is on the flight sheet, minimizing the time spent looking for entries. A sample of the first entry is then poured and placed in front of each judge. The steward tells the judges what the entry number is, the category, and subcategory. The second entry is immediately poured and handed to each judge. This process is repeated until all the entries have been served. Once the judges have sampled all the entries in silence and jotted down thoughts if desired, discussion starts. Discussions in the BOS round can be lively as judges do not always perceive the same things or agree on what is medal-worthy and what is not.
Stewards, organizers, and other judges should not interfere with BOS judging. Competitions may choose to hold the BOS round in a different room to avoid interference. Try to accommodate requests of the judges. If they feel distracted by those around the table, clear the room to help them finish faster. Some judges are not disturbed by bystanders, as long as they do not interfere with the judging. Respect their wishes. However, stewards and staff assisting the competition should not interfere with the judging, either. They may answer direct questions, such as information about certain beers, or whether a top pick is required, or if additional awards will be given. Stewards for the BOS round should not include those with entries on the table.
There are two basic ways of reaching consensus on winners. One is to begin by eliminating entries that the judges feel are less ideal. When the pool of remaining entries has been reduced to a manageable number, the panel begins to discuss the best entries, generally their top three. The other process skips the elimination phase and immediately advances the best entries. Either way works fine and BOS panels should be allowed to decide for themselves how they want to proceed, including devising any method of their choosing.
As agreement is reached that a particular entry is no longer needed on the BOS table, when individually or as the group that are not among the judges’ favorites, the glasses are dumped unless the judge chooses to set it aside to drink later, or offers it to stewards or other judges in attendance. Removing cups as the entries are knocked out helps the judges know at a glance how many entries are left for consideration.
By the end of the BOS round, the best entry and any runners up are determined. This same process applies for determining BOS meads and ciders if these categories are being judged separately from BOS beer, except that one or two stewards would be sufficient since the number of BOS entries is generally significantly less for meads and ciders. Another option for a BOS round with only a few entries is for the judges to open the bottles and pour for themselves.
Once the BOS winners have been determined, an award ceremony is frequently held. The “ceremony” consists of announcing the winners to those gathered and presenting awards to winners in attendance. If awarding ribbons, it is helpful to have the backs all filled out except for the BOS ribbon, before the end of the BOS so that only the BOS ribbon needs to be completed. Doing this keeps the competition rolling along smoothly. Creating and sticking printed labels with the brewers’ name and judging category on the backs of the ribbons instead of handwriting the information speeds up the process.
BOS and Other Awards
Some competitions have multiple awards. If one of these awards is selected by a method other than a traditional best-of-show panel (e.g., people’s choice, local favorite, brewer’s pick, etc.), this panel must be separate from the traditional best-of-show process. Winners may be selected from the BOS entries, but this separate selection process must not interfere with or otherwise bias the BOS selection.
For example, sometimes a beer will be selected by a professional brewer to be brewed at their facility. That brewer may be selecting a beer based on what they are able to produce on their system, the cost of the beer, or how the beer might overlap their current portfolio. These are not traditional criteria used by best-of-show judges who select beer based on quality and stylistic fidelity. The fact that a professional brewer (who may or may not be a BJCP judge) prefers a specific beer should not affect the traditional BOS selection process. Likewise, voting for a people’s choice award should not be known to BOS judges prior to their making their selection.
At Day’s End
When the judging and award portions of the competition have been completed, the cleanup remains. The extent of the cleanup to be done by competition staff should be worked out with the facility prior to the competition. Should the entire responsibility for cleanup fall on the competition staff or club, there is a large amount of work yet to do.
All competition scoresheets, ribbons and prizes not awarded should be stowed safely in designated boxes or storage bins/crates. Used cups should be recycled or discarded, pitchers emptied, dump buckets rinsed and stacked, stale bread/matzos/crackers thrown away, competition supplies collected and put back in storage bins, scrap paperwork recycled, and unused competition paperwork stored for future competitions. If necessary, trash and recycling cans should be emptied into designated receptacles.
Empty bottles can be rinsed out, if possible, and recycled or put in case boxes for participants to take with them and reuse, or both. Full bottles remaining in the cooler must be removed, given away to judges, or saved for a variety of purposes at club meetings. Another option is to open them all and empty them, but this a lot of extra work and a waste of good homebrew.
Depending on the agreement made with the facility, tables may need to be collapsed and tables and chairs stacked. If there is an outstanding bill for food or beverages consumed during the competition, payment should be made unless other arrangements have been agreed upon. Before leaving, a walk-through should be completed to make sure that the room is presentable and nothing has been left behind.