TopCoder Crowdsourcing

?
?
?
?
?
?
?
10 BURNING QUESTIONS ON CROWDSOURCING:
YOUR STARTING GUIDE TO OPEN INNOVATION AND CROWDSOURCING SUCCESS.
THIS BOOK IS DEDICATED TO THE TOPCODER COMMUNITY. YOU NEVER CEASE TO AMAZE US.
EVERY GRAPHIC, ICON, COPY FORMATTING, AND PAGE DESIGN YOU ARE ABOUT TO EXPERIENCE IN THIS BOOK, AND CONVERSION WIDGETS AND BANNERS SEEN ON TOPCODER.COM WERE CREATED BY THE TOPCODER COMMUNITY VIA OPEN COMPETITIONS. DURING THE PROCESS OF WRITING, EDITING AND PREPARING THIS BOOK FOR PUBLIC CONSUMPTION, WE LEVERAGED TWO EXCEPTIONAL COPILOTS AND ONE WITH A FOCUS ON DESIGN ELEMENTS AND ONE WITH A FOCUS ON WEBSITE IMPLEMENTATION AND CONVERSION. THIS BOOK IS DEDICATED TO ALL THE AMAZING AND TALENTED INDIVIDUALS WHO HELPED BRING THIS BOOK TO LIFE.
10 BURNING QUESTIONS ON CROWDSOURCING
PAGE01
INTRODUCTION
?
?
WELCOME, JUMP RIGHT IN!

So you and your enterprise want to use Crowdsourcing and Open Innovation to accelerate innovation cycles and increase productivity - great, we don't blame you! Open Innovation platforms implementing Crowdsourcing techniques are proving time and again to be an exceptional way to produce amazing assets and solve complex problems. In short, this is an exceptional way to innovate, but like all else in business and in life, you will have more success if you are prepared and understand what you are after. Perhaps you have read a case study or two, or otherwise heard about the successes and promises of Open Innovation and Crowdsourcing and it piqued you, so that you now nd yourself downloading and reading this eBook. Well, we thank you, welcome you, and commend you for taking this next step. This starting guide is about your path, your understanding, and your ultimate success in Open Innovation and Crowdsourcing.
PAGE02
INTRODUCTION
WHAT YOU ARE ABOUT TO ENJOY
THE STARTING GUIDE BREAKDOWN

THIS STARTING GUIDE IS BROKEN DOWN INTO 3 PARTS, WITH 10 CHAPTERS IN TOTAL, ONE CHAPTER DEDICATED TO EACH BURNING QUESTION. THE 3 PARTS ARE:
LOGISTICS - HOW DOES CROWDSOURCING LOGISTICALLY WORK AND WHAT MUST I DO TO GET STARTED? ? EXECUTION - HOW DO I PRODUCE IN OPEN INNOVATION AND CROWDSOURCING?
GROWTH HOW DO I SCALE OPEN INNOVATION AND TRANSFORM YOUR ENTERPRISE?
In total, this starting guidebook will answer 10 burning questions on Open Innovation and Crowdsourcing that every person (and enterprise) venturing into this world of massively parallel production should understand before they get going. Each answer will explain the importance of the question why it matters to you and the specic way in which that question is answered for the TopCoder Enterprise Open Innovation (EOI) platform. The world has shifted. There is a global talent pool with exceptional and specic skill-sets waiting for their opportunity to produce for you. Through Open Innovation and Crowdsourcing, you can innovate and deliver like never before and we hope this starting guide is your launching pad to amazing success.
PLEASE ENJOY THE 10 BURNING QUESTIONS ON CROWDSOURCING AND BEST OF LUCK ON YOUR CONTINUING JOURNEY TO PERSONAL AND ENTERPRISE SUCCESS!
PAGE03
TABLE OF CONTENTS
PAGE05
PART 1: LOGISTICS - HOW DOES CROWDSOURCING

LOGISTICALLY WORK AND WHAT MUST I DO TO GET STARTED?
How do I decide how much money to offer the crowd? How long should my contests run for? How do I decide and pick the winners?
PAGE 13
PART 2: EXECUTION - HOW DO I PRODUCE IN

OPEN INNOVATION AND CROWDSOURCING?
How do I prepare a project for success? How do I ensure the right talent is working on my project or build? How does iteration and subject matter expertise work with "the crowd"? How do I manage my Open Innovation & Crowdsourced competitions?
PAGE26
PART 3: GROWTH HOW DO I SCALE

OPEN INNOVATION AND TRANSFORM YOUR ENTERPRISE?
How do I accomplish complex as well as simple tasks? How do we protect Intellectual Property? How can we truly scale Open Innovation?
PAGE04
PART 1 - LOGISTICS HOW DOES CROWDSOURCING LOGISTICALLY WORK AND WHAT MUST I DO TO GET STARTED?
You host a competition and you get the winning solution, idea, or digital asset Whats so hard about that!?
Yes, it can be that easy. Before we dive into any and all of the forthcoming Q&A please realize one over-arching point: The platform and community/crowd you choose to engage makes all the dierence. Seek a platform that is truly focused on your outputs and your innovations; one that doesnt ask you to get bogged down in the minutia, but instead allows you to focus on your productivity and your value creation. The community of solvers, producers, and contributors you engage with Open Innovation and Crowdsourcing competitions should be both wide and deep, meaning you should be able to accomplish a wide variety of work via your community, while accessing hyper-specialized talent with very specic skill-sets that compete for you in each unique competition, contest, or challenge.
TopCoder focuses on digital asset creation through competitions and we canvass an array of specic competitions in that space. Other communities that leverage Crowdsourcing focus on things such as video production, localization, and physical mobile application testing on specic handsets. Understand your needs, do your homework, and then choose wisely. Regardless of which communities you work with in the future, the following content will help you prepare for repeatable success.
Lets dive into the questions.
PAGE05
BURNING
QUESTION #1
HOW DO I DECIDE HOW MUCH MONEY TO OFFER THE CROWD?
$
?
Why this question matters to you:
Crowdsourcing and Open Innovation platforms are an evolved marketplace and economic factors are in play, meaning you need to correctly price tasks, work, and challenges in order to draw-out the best talent from any given community so that they will work on your projects. One of the major benets of Crowdsourcing is the fact that it is mainly meritocratic, meaning the lions share of the prizes goes to the winning contributors and solvers and you pay for the best outputs, not eort. So you have to make sure youre oering a fair market value for the lift you are expecting of the individuals within the community. Remember, they the community members creating value for you - are taking on risk and very often have no guarantee of payment, so pricing a contest fairly is crucially important to overall success. At the same time, you need to work within your budget. Dierent Crowdsourcing and Open Innovation communities have an incredible variety of payments being oered spanning from literal pennies for the completion of a micro-task to prizes of over a million dollars being oered for solving really large, complex problems and algorithmic challenges.
PAGE06
SO HOW DO YOU DECIDE HOW MUCH MONEY TO OFFER THE CROWD FOR EACH SPECIFIC COMPETITION OR CHALLENGE?
?
ANSWER:
You shouldnt. This is part of the minutia of Crowdsourcing we discussed earlier and it should not be a focus of yours. Of course you have budget, but setting prize monies for specic challenges should be based on historical success of similar challenges on whichever platform you are using. You should be working on a platform that oers you clear and transparent visibility into the market price for each specic type of task or challenge.
Specically on the TopCoder EOI Platform:

TopCoder has been running atomized challenges and contests for more than a decade. Our metrics (based on thousands & thousands of successful competitions), user dashboards with transparent access to current market analytics, and Platform Specialists (a client resource that helps our clients get the very most out of the TopCoder Community and Platform) suggest the right level of pricing for each of your specic competitions. We know what pricing level works best and for what type of competition. You of course have nal say and must green-light any given project or competition, but whether it be something small like an icon design contest or something quite large such as a Big Data long-form algorithmic challenge, the TopCoder Platform history will guide you. On our platform, your focus needs to stay on the results and the value you can add by helping the community create for you, not lost in the weeds of deciding what prize monies will be most eective for your particular contest. Stay focused on your success, you will hear this theme a few more times throughout this guide.
PAGE07
BURNING
QUESTION #2
HOW LONG SHOULD MY CONTESTS RUN FOR?
?
?
?
Why this question matters to you?

Open Innovation and Crowdsourcing isnt about running one
?
?
stand-alone challenge and being done with it. Instead, its about transforming how you approach the very notion of work, how you access global talent, and ultimately, how you produce, innovate, and succeed like never before. To achieve this long-term success, it means making Open Innovation part of your daily routine, and for that to happen, you have to have an understanding of how long things will take to get done and delivered back to you. You have deadlines, people to answer to, and projects that have to cross the nish-line on time. From the standpoint of the community or crowd you are working with, contests that are either too short or too long in duration hinder the maximum participation that will drive the best results.
SO IN SHORT, KNOWING HOW LONG YOUR CONTESTS NEED TO RUN FOR IN ORDER TO BE SUCCESSFULLY COMPLETED BECOMES A CRUCIAL PIECE OF YOUR SUCCESS IN CROWDSOURCING AND OPEN INNOVATION.
PAGE08
HOW LONG SHOULD MY CONTESTS RUN FOR?
?
ANSWER:
It greatly depends on the type of work you are putting out to a community and crowd, and much like the rst question; you should be seeking guidance from the operators of the platform. You shouldnt be worrying about how long a particular contest is going to run for. Instead, your focus should be on the entirety of the project, and orchestrating multitudes of projects in unison to vastly increase your personal productivity via Open Innovation and Crowdsourcing. In short, its not about how long, its about how predictable and repeatable a certain type of contest is, so you can eectively set a contest up for success, walk-away from it while the competition is on-going and doesnt require your active involvement, and in the meantime, work on other projects, pieces of projects, and yes, even run more competitions to really force-multiply your eorts.

The very vast majority of competitions on the TopCoder Platform with the exception of Bug Races that are a rst to x type contest are strategically time-boxed, meaning they have a pre-determined and set duration for the specic type of challenge. For instance, a Wireframe competition will typically last between 2 4 days per round of active competing, while a System Architecture or a UI Prototype competition will last around 10 days. Again, the more important factor is the predictability you are granted. When you trust you will receive quality and innovative work by the set deadlines, you can work smarter, more eciently, and more eectively.
Do TopCoder competitions sometimes need an extension?

Of course, it does happen, but you can rest assured managing time extensions is sincerely easy and most often, it wouldnt be you needing to make the adjustment, your Copilot which we will cover during the growth & scaling portion of this book - can handle this for you.
PAGE09
BURNING
QUESTION #3
HOW DO I PICK THE WINNERS?
?
?
?
Why this question matters to you?

?
This is all about your time and understanding your responsibilities. If
?
[]
[]
?
?
you and your co-workers indeed must be the one choosing the winners, it behooves you to set aside time to do this extraordinarily important task. That time should be factored into your work schedule and this responsibility is something to take quite seriously after all, the winning submissions are your outputs! There are times when choosing a winner should not be your responsibility, for example if you are too busy, or would like technical experts to serve as judges. From the standpoint of the community or crowd you are working with, judging has to be absolutely transparent, fair to the members, and when appropriate, standardized. To keep a community engaged and producing creative outputs, fairness and consistency in judging is paramount. Selection of winners must happen on expected schedule so that the individual members can most eectively manage their projects and workload.
[]
UNDERSTAND YOUR RESPONSIBILITIES. IF YOU ARE DECIDING A WINNER, DEDICATE TIME TO YOUR SELECTION PROCESS AND ALWAYS TREAT THE COMMUNITY OR CROWD MEMBERS FAIRLY. STICK TO YOUR PRE-DETERMINED SCHEDULES AND FINISH YOUR CONTESTS ON-TIME.
PAGE10
HOW DO I PICK THE WINNERS?
ANSWER:
Depending on the type of competition, your involvement in choosing the winners will vary tremendously. Why? Well, if it is a subjective contest, such as a design competition, it is very likely you would prefer to choose the design you and your team feels is the best t. Why would you hand over that decision to anyone but your team? Perhaps you might see benet in doing so to have an outside team pick their favorite to see how it compares to your choice, but ultimately, the nal decision is likely best to remain with you. Then there are contests where the outputs can be measured against a certain set of pre-determined criteria. For instance in a data compression competition, you simply might be looking for the solution that performs the greatest while retaining a certain level of identied data integrity. Here, the winner would simply be decided by the performance of the solution.

TopCoder canvasses 3 distinct Pillars of Digital Creation including FEI or Front-End Innovation, Software Development, and Analytics & Algorithms. Lets quickly visit your responsibility in each of these pillars as it pertains to selecting winners.
Selecting Winners on Each Track
FRONT-END OF INNOVATION
SOFTWARE DEVELOPMENT
ALGORITHM ANALYTICS
SUBJECTIVE (Client Decides Winner)
OBJECTIVE (Rigorous Scorecards Decide Winner)
PERFORMANCE (Best Solution against pre-determined metrics wins)
PAGE11
Front-End Innovation (FEI) deals with mainly subjective contests where design and user-experience are the focus. These can be idea or conceptualization contests, UI/UX innovations, storyboards, icons, click-able prototypes, and more. For these types of contests you and your team are responsible for selecting your favorite submissions. Software Development is completely dierent. Deciding between a better piece of Java code could engulf your entire day. Instead at TopCoder, each piece of software developed via competition is peer reviewed by 3 community members who utilize didactic and language specic scorecards. These scorecards - and what they seek to grade - can (and often are) edited to t a specic clients needs. The work is graded, the top rated piece of code wins, any remaining aws are documented, xed by the winning competitor, and the nal output is your deliverable. Algorithms & Analytics is most often completely objective. Well before an algorithmic competition or Big Data challenge commences, our team works with you to highly dene the parameters of success and how the solutions will be graded against one another. At the end of the specied competition period, there is a clear winner that simply performed the best within the parameters pre-described. There is one exception in this pillar. We also host Big Data ideation competitions that ask our global community to provide ideas on how they would utilize a large data set and the assets and applications they would seek to create from the data. This is more of a Front-End exercise and therefore the outcomes are subjectively evaluated and judged.
LET'S DEFINE IT WHO ARE PEER REVIEWERS AT TOPCODER?

Reviewers at TopCoder are community members who have been elevated to this position of status within the community. They are oered this position based on consistently strong work and must retain a certain level of competition and success in that domain to remain a peer reviewer. We utilize peer reviewers for software development competitions.
PAGE12
PART II: EXECUTION HOW DO I PRODUCE IN OPEN INNOVATION AND CROWDSOURCING?
Yes! You have now decided to utilize Open Innovation and Crowdsourcing to succeed like never before!
You now have a better understanding of how it all works. But understanding why and executing how are quite different. This part will concentrate on your execution and how your consistent approach will lead to repeatable success in Crowdsourcing and Open Innovation. As you read through this section of the starting guide, think about how good habits are formed and how consistent execution becomes a reality. No matter what you are executing on, consistent success doesnt happen overnight and it certainly doesnt come to be without effort and practice. Treat your Open Innovation and Crowdsourcing execution in the same light. Mastery of these tactics and techniques are not a singular event, but rather a process of learning, experimenting, and achieving.
PAGE13
BURNING
QUESTION #1
HOW DO I PREPARE A PROJECT OR A SERIES OF PROJECTS FOR SUCCESS?
?
?
?
?
?
?
?
Lather, rinse, repeatable success: Crowdsourcing and Open Innovation shouldnt be about running one isolated competition, never to revisit the methodology again. If that were the case, you probably would not have decided to download and digest this starting guide. This question matters because as you master the art of Open Innovation you will learn from your own experiences, what works best, what can save you time and ultimately, how it can help you predictably & repeatedly steer Crowdsourced competitions to resounding success. After all, thats why youre here to succeed. From the standpoint of the community or crowd you are working with, preparation of the inputs heading to the members is crucial. Vagueness, unclear goals, a lack of necessary shared domain, and other negative factors can sincerely disrupt your contests and impact your results. Be comprehensive. Be clear, and anticipate questions the community or crowd will likely ask, and prepare those answers in advance.
PAGE14

HOW DO I PREPARE A PROJECT OR A SERIES OF PROJECTS FOR SUCCESS?
ANSWER:
In the world of digital creation, developing well written, appropriately detailed specications is paramount to success. First and foremost, ingratiate yourself with how the community or crowd you are about to engage is accustomed to receiving information. Look at other examples of inputs that community has digested in the past and ask specic questions as to why certain inputs seemed to be favored over others. Secondly, understand the right level of detail you will need to provide and for which type of contest. If you are seeking solutions that will be judged subjectively, you should purposefully keep the specication or contest input more obtuse and allow for the natural creativity of your community to arise. Part of Open Innovation and Crowdsourcing success is your willingness to be surprised! Then there will be other competitions where rigorous documentation and knowledge share of a specic domain is simply essential to procuring a highly valuable contribution from the community. Think hard about how much needs to be shared, prepare that documentation in a way the community is accustomed to receiving it, and make yourself or a subject matter expert from your team available to answer questions that will come from the community. Finally, ask your platform provider if they oer help in creating the specication. Youll read in the TopCoder EOI section (just below) why this might save you a great deal of time and eort, and result in superior outputs as well.

Everything detailed above is solid advice. We see idea competitions launched routinely where our clients purposefully keep the input to 2 to 3 short paragraphs because they desire the widest lens of potential ideas and solutions. Conversely, we host Big Data and algorithmic challenges where the problem set is dened in amazingly specic ways and the input into the community is quite a large set of data and highly-dened specications. The specication needs to match the desired action. But there is another, and potentially more important, piece for you to understand about the TopCoder Platform and specications.
MANY OF OUR CLIENTS PREFER TO NOT CREATE THEIR OWN SPECIFICATIONS FOR CONTESTS THEY ARE RUNNING ON TOPCODER. SO WHERE DO THE SPECIFICATIONS COME FROM?
TopCoder has contests where the output is the specication document. This allows our clients to really stick to strategy & execution and to avoid getting bogged down in transforming real world projects into atomized competition inputs. Some clients prefer to create their own specications while others utilize these specialized contests to produce the documentation for them. It is your choice.
PAGE15
BURNING
QUESTION #2
HOW DO I ENSURE THE RIGHT TALENT IS WORKING ON MY PROJECT OR BUILD?
?
?
?
?
?
?
?
The right talent will most often procure the right results. Open Innovation and Crowdsourcing are focused on the outputs the competition will breed. The more specic the task or project - the more hyperspecialized the talent you will want to attract to the work. Attracting 700 coee-shop baristas will likely do you little good if the project is to rebuild a 6Os era muscle car engine. If you are not engaged with the right community (or crowd) for the work or innovation you seek to accomplish, your chances of success diminish greatly. It should be noted: There is a huge dierence between the need to recreate something that already exists a 60s era muscle car engine versus something that has yet to be solved or created. When the object of the contest is more open because no solution yet exists, we often see that near-eld repurposing of knowledge and skills can play a crucial role in helping to formulate the new solution. From the standpoint of the community or crowd you are working with, the individuals that comprise the crowd have very specic skill sets and personal desires. Put simply, they really enjoy doing certain types of work that they are either already good at, or want to learn and master. But like everyone else, their time and bandwidth (individually) is limited, so they will gravitate towards work they enjoy, work that is rewarding both nancially and personally and, work that ts their personal bandwidth, meaning the size of the ask is a crucially important factor.
PAGE16

HOW DO I ENSURE THE RIGHT TALENT IS WORKING ON MY PROJECT OR BUILD?
ANSWER:
Depending on which community, or crowd, or network of solvers, or platform you choose to engage will determine greatly your ability to repeatedly access the hyperspecialized talent you need in order to succeed. Sure, if you can come up with an amazing dollar prize amount and can craft a contest so worthy that all the technology and social blogs are sharing the challenge for you; that could also do the trick. But, the big dierence is, that path is nearly impossible to scale and again, youre not here to run one competition. Rather, your focus needs to be on the repeat-ability of the process, so that you can drive extraordinary value and outcomes for your team and potentially transform how your enterprise approaches innovation and productivity.

It begins and ends with our amazingly skilled and continuously growing community. TopCoder is a community of now over 485,000 designers, developers and data algorithmists. Besides being a large community made up of mainly hyperspecialists; the TopCoder platform itself is equipped with a prediction algorithm that alerts you, the user of the platform, if your scheduled contest is not attracting the right-level of talent to your specic contest. How do we know? We measure a litany of performance metrics for all members, not just ranking their skills in specic disciplines, but also tracking their individual reliability ratings and myriad other factors. On the occasion when the platform is predicting a less than optimized result, your contest will not fully launch and youll be asked to revisit your specication or monetary oering. Perhaps the problem wasnt well enough dened or perhaps its more complex than you think and therefore a small increase to the prize money oered is in order. On the TopCoder platform you will not have to worry, or spend time attempting to attract the right talent to your work. The platform handles this for you.
LET'S DEFINE IT WHAT IS NEAR-FIELD REPURPOSING?

This occurs when a person who is highly trained in a skill or trade applies a specic piece of knowledge they have learned practicing in their domain over to another eld. The individual is repurposing what they already know to help solve a problem in a eld they have little or no professional experience in. Often, repurposing of knowledge brings a fresh approach, or idea, or solution to a challenge that in-eld experts simply would not have attempted or even thought of trying.
PAGE17
BURNING
QUESTION #3
HOW DOES ITERATION AND SUBJECT MATTER EXPERTISE WORK WITH THE CROWD?
?
?
?
?
?
?
?
It may be high time to debunk a myth. Crowdsourcing and Open Innovation are not magic. You do not simply post a contest, only to ignore the contest for a set period of time, and receive amazing results in return. It just doesnt work that way. You need to be aware that Open Innovation and Crowdsourcing do take eort and specic eort at that. You will be involved in the success of your Crowdsourced projects and in at least two very specic ways providing iterative feedback and managing your internal subject matter experts, ensuring knowledge ows properly from your internal organization out to the community you have decided to engage. Be prepared to engage work dierently, but still be prepared to work! From the standpoint of the community or crowd you are working with, iteration with the client is a crucial step to their individual success. Remember, the community member is here to succeed too! Their eorts and their ability to create outstanding assets often hinges on receiving quality feedback from the client. Regarding access to subject matter experts; if a particular challenge is complex and the necessary domain knowledge is not shared from your team to the community or crowd, you likely will nd participation quite light. The converse is can also be true. If domain is properly shared, the community will likely respond in-kind, driving participation and the volume of quality submissions higher.
PAGE18

HOW DOES ITERATION AND SUBJECT MATTER EXPERTISE WORK WITH THE CROWD?
?
ANSWER:
Iteration: Open Innovation can be quite collaborative and iterative if you approach the work properly and you are engaged with a community that fosters such activities. Often while running an Open Innovation competition you are going to be presented outputs that surprise you. It could be a nuanced and clever idea, a really sharp icon design, or a new approach to a solution your team would likely never have come up with. Very often, outputs, as they are rst presented, are not nished - or perhaps better put they can use some level of renement and editing. This is where iteration comes into play. If you are able to oer feedback and specic guidance to competitors during a competition, you should. Be clear, be honest, be respectful, and communicate what is succeeding so far, and which areas of the design or solution you feel can be improved. Take this role of providing critical feedback very seriously. You will be amazed how well competitors will take good direction and how advanced their new outputs will be because of the quality feedback you provided. Subject Matter Experts: Depending on the type of challenge or contest you are seeking to run, you may need to consider how you will eectively transfer knowledge to your community or crowd so that they can eectively work on your problem or task. The greatest step you can take is to properly facilitate this knowledge share. Prepare your internal team well before hand and alert the specic players who may need to be involved that they will be required to spend time either presenting or answering questions in which they have a certain domain expertise. Ask the community or Open Innovation provider before hand how this is handled and press for a detailed answer. If you cant eectively share knowledge outwardly with a community or crowd it might prohibit the range of work and innovation you can attempt.
PAGE19

Iteration is a key component to success for many TopCoder competitions. As you might imagine, the more subjective contests that favor design and user experience are where we see the highest level of iteration. In our design (Front-End Innovation) contests we routinely have what is called a Checkpoint. This is a simple pause, typically near the middle of the competition where the competitors initial submissions are reviewed, feedback is generated by the client, and then delivered back to the members so they can digest the critique and make the adjustments they believe will help them win the competition. Subject matter experts can also be crucial to a contests success. Often at TopCoder, we will have clients who want our community to work in a specied technology environment or perhaps work on a specic algorithmic challenge where domain knowledge can play a pivotal role in shaping their attempts, and thus their outputs. When this is the case, it is imperative that we foster the lines of communication as much as possible between an internal teams subject matter expert(s) and our community. Through the use of scheduled forum sessions where subject matter experts participate in a highly active Q&A exercise with interested community members our clients and competitors engage in collaborative questioning and the necessary knowledge sharing takes place. We augment these collaborative sessions with the needed written documentation so our community can access additional information they feel they might need in order to compete at a high level.
LET'S DEFINE IT WHAT IS COLLABORATIVE QUESTIONING?

When domain expertise is being shared in an open forum session on the TopCoder Platform, each participant has the advantage of hearing answers to questions they themselves may never have thought to ask. Collaboratively, the community asks scores of questions. Individually, they benet from the answers and knowledge transfer taking place and that can assist them in their pursuit to create a better solution for you.
PAGE20
BURNING
QUESTION #4
HOW DO I MANAGE MY OPEN INNOVATION & CROWDSOURCED COMPETITIONS?
?
?
?
?
?
?
?
You can master managing Open Innovation competitions just as you can master managing an internal team or external outsourcing partner. Yes, the work itself is dierent, but as we discussed earlier, there is still work to do Crowdsourcing is not magic! If a process of managing the work breaks down, it doesnt matter what methodology you are incorporating to get the work accomplished, you will greatly limit your success. However, if you learn to manage Open Innovation contests eectively, allowing you to have assets created in a massively parallel fashion, you will realize success and productivity on levels you never thought possible. From the standpoint of the community or crowd you are working with, individual members like to participate in contests and challenges where the management of the contest is professionally executed and consistent. Community members will also gravitate towards your specic challenges and contests once your history and reputation intra-community is established.
PAGE21

HOW DO I MANAGE MY OPEN INNOVATION & CROWDSOURCED COMPETITIONS?
? ? ?? ? ? ? ?
?
ANSWER:
answered and when you can, anticipate what they will ask for and provide it at the onset.?
Some of the below has already been canvassed in this starting guide, but it bears repeating. After preparing your contests for success, by ensuring the right level of detail has been shared and that you have attracted the right level of talent to your work or challenge, your focus should shift to managing the contest(s). So where should you start? First, you will need to learn to let go. Remember you or no longer doing the work and your role has therefore shifted. Instead focus on making sure your community or crowd has everything they need
Second, when questions arise from your community about a particular project or task and please note they will arise and this is a good sign of healthy participation answer your potential contributors fast and as thoroughly as possible. Remember, it is highly probable that people within your community or crowd speak English as a second language. Dont be afraid to over-explain details, this will help you avoid any miscommunication and time wasted because of it. If you have visuals, screenshots or picture examples you can share, do so if it helps to more eectively communicate your answers. Be clear! Finally, learn to work in a massively parallel fashion. We will discuss scaling Open Innovation later in this starting guide. For now, understand that you should challenge yourself and seek to conduct several contests at one time. Why? This will force you to not revert back to bad habits of doing the work and instead ll your time with orchestrating multitudes of solutions. This is where you want to be. You are a conductor of grand value, so emerge as the maestro by challenging yourself to successfully manage multiple contests at one time. When you arrive at this point, your value soars, you accomplish more, and you successfully innovate faster than ever before.
PAGE22

There are two signicant things to know about the TopCoder platform. First, it is actually a platform in that it is constructed so that you can launch, manage, and nish Open Innovation & Crowdsourced contests on-demand. Below are few snapshots from what would be your dashboard view which we call Cockpit. All of your contests, all of your Copilots (more on their role in just a moment), your budgets, timelines, forums and more, neatly laid out for you so you can manage multiple competitions simultaneously. It is a true productivity platform. Again, Open Innovation for your business or enterprise is not about running one competition, but rather it is focused on force-multiplying your teams capabilities to achieve much more production in what becomes a continuous innovation cycle. To achieve this level of repeatable success we created and deliver Cockpit to every client and pair the platform with the training needed so you can get going fast, ramp-up your production over time, and achieve grand scale as you master Open Innovation management.
PAGE23

TopCoder Cockpit Dashboard
TopCoder Cockpit Project Analytics
PAGE24

We just introduced the terminology of TopCoder Copilots. Lets dene their role and how they help you execute Crowdsourced competitions on-demand. Copilots are your key to scaling your individual eorts. They manage the technical aspects of crafting, launching, and managing competitions all the way through successful delivery. They do the technical heavy lifting and manage the process of Crowdsourcing so you can concentrate on strategy and scale. As they answer forum questions and keep your contests on track for success, you focus on the orchestration of all of your competitions with a keen focus on delivering timely feedback and the needed information that results in cleanly written specications. Envision yourself, directing 3 or 4 Copilots at one time, and they launching 2 or 3 competitions for you each in parallel. Now, you are orchestrating 6, 8, or even 11 contests at once, each set to deliver back to you a valuable asset. That is we call force-multiplication. MASSIVELY PARALLEL PRODUCTION OF INNOVATIVE ASSETS
Assets Copilot Competitors Architecture Assembly Testing UX Idea Gen Rapid Prototyping Big Data Challenge Algo Optimization Storyboard Wireframes Concept
Now envision several of your most productive team members multiplying their outputs and delivering for your enterprise or business at 3X, 5X, or even 10X their traditional productivity levels. Do the work necessary to achieve this type of scale with your team and your value as a strategic leader who delivers results will soar.
LET'S DEFINE IT WHO ARE TOPCODER COPILOTS?

Copilots are an elite pool of TopCoder Community members who have proven to be exceptional technical managers of the TopCoder Platform. They help you through all phases of competition from organizing details for proper specications, to answering community forum questions, to oering their feedback on what they deem are the strongest solutions - and they are incentivized on the successful completion of the contest. They are solely focused on your success for each contest.
PAGE25
PART III GROWTH HOW DO WE SCALE OPEN INNOVATION AND CROWDSOURCING AND CONTINUOUSLY CREATE VALUE?
Is Crowdsourcing a nice marketing tool to get eyes to your work or is it transformational, allowing you to innovate faster and produce more, while taking on less risk?
Its really up to you. Today, global communities and production platforms exist and your willingness to re-think work and push the boundaries of what can be accomplished via Open Innovation will determine how much you can get done. This part of your starting guide is all about your growth, your innovation, and your success. Individually, you are likely looking to grow your skills and emerge as an innovation expert within your enterprise or business. That emergence can be your reality and you can achieve amazing growth. Build on all that you have learned so far and begin to think about real world projects and scenarios you can envision beneting from Open Innovation. Think about how you would structure these projects, the subject matter experts you might bring to the community or crowd, and more in order to achieve success. As you read through this nal part of the starting guide, begin honing in on the details and thinking through them in your mind, or better yet, mapping them out visually.
PAGE26
BURNING
QUESTION #1
HOW DO I ACCOMPLISH COMPLEX AS WELL AS SIMPLE TASKS?
?
?
?
?
?
?
?
Are you here to transform how you approach productivity and innovation or are you here to source more aordable ways to accomplish fairly mundane tasks? Ask yourself this honest question. If your answer is the former, than how much you can accomplish via the methodologies matters tremendously. From the standpoint of the community or crowd you are working with, individual members tend to be attracted to challenging and complex work if it is well organized and properly dened. There are myriad reasons why, ranging from greater potential prize money earned, their desire to work on challenges their daily life does not present to them, their want to solve something that has no existing solution, community peer recognition, and more can motivate a community member and get them excited to help you succeed.
PAGE27
HOW DO I ACCOMPLISH COMPLEX AS WELL AS SIMPLE TASKS?
ANSWER:
Todays Crowdsourcing and Open Innovation landscape is maturing rapidly. Sure, there are ample solutions and crowds that now exist that can help you do a mundane task; such as physically waiting in line at the DMV for you, or localization services that utilize crowds to translate documents. These can save you time and money. They can make you more ecient, but not more innovative. The other side of Crowdsourcing encompasses projects, contests, and challenges that are truly innovative. Where an unknown or optimized solution is what you are seeking, and it doesnt exist yet! These types of challenges and contests are more properly dened as Open Innovation.
To recap this important distinction:

The output of a Crowdsourced task is the accomplishment of the task itself, likely done at some cost savings and potentially much faster. The focus is on eciency. The output of an Open Innovation contest is a new (or newly optimized) solution or asset that did not previously exist. The focus is on innovation.
Crowdsourcing & Open Innovation
Micro-tasks/Crowdsourcing
Cost Savings Faster Results Free-Up Your Internal Resources Focus on Higher Value Task
Open Innovation
New Products Breakthrough Discoveries Extreme Value Outcomes Community Building and Engagement
Bottom Line Efficiencies
Top Line Growth
What are you trying to accomplish?

In todays market, you have access to communities and crowds that will help you accomplish both.
PAGE28
At TopCoder we have over 30 specialized competitions ranging from the very front-end of innovation contests focused on ideas and concepts all the way through Bug Hunts and Bug Races contests to identify, and then x veried bugs. As a client you can choose to do a less complex singular competition like a design contest and then bring that asset back to your team to continue the work internally OR you can build a digital asset all the way through the software development life-cycle from ideation through testing & maintenance. It is your choice. This choice is made possible due to TopCoders process of atomization and one of the consequences of atomization is incredible exibility you gain throughout the development life-cycle. At TopCoder we have delivered solutions to complex challenges and projects ranging from Big Data Bio-medics contests, mobile applications to be used in remote and desolate environments, energy harvesting for the International Space Station, legacy system modernization and beyond.
LET'S DEFINE IT WHAT IS ATOMIZATION?

If you were to dissect a sleek iPad app and break it apart into the digital pieces that make up the whole working unit and the work that had to happen prior to nal digital assembly, you would see highly specic accomplishments in areas such as icon design, user interface design, software engineering, application architecture, wireframes, storyboards, prototypes and more. At TopCoder, when we take on any project, simple or complex, we atomize the project into its smallest units of work. Each piece of the whole is hosted as a competition. That process is atomization.
PAGE29
BURNING
QUESTION #2
HOW DO WE PROTECT INTELLECTUAL PROPERTY?
?
?
?
?
?
?
?
Much like we discussed earlier, if you are highly limited in what you can attempt and therefore create via Open Innovation, then the potential value to you and your enterprise diminishes. And that is not why you are here. Protecting intellectual property is a legitimate concern most every enterprise will have when assessing Open Innovation. If it is not your concern, it will be someone elses from your team. Your best strategy is to understand the risks, the process, and the platform you are engaging and make educated decisions based on your understanding. From the standpoint of the community or crowd you are working with, the smaller the barriers to entry to participate in any given contest or challenge, the greater the participation rates will likely be. However, individuals that comprise a community typically understand that additional layers of screening or an additional security process sometimes is necessary so that they can work on what is deemed by the client as an IP sensitive project.
PAGE30
HOW DO WE PROTECT INTELLECTUAL PROPERTY?
? ?
ANSWER:
??
This question is impossible to answer in a generic fashion as we have in this section throughout this starting guide. TopCoder does not want to begin to profess how other platforms and communities handle this sensitive subject. What we can communicate generically is that everything has some level of risk. Whether it is traditional hiring, outsourcing, or Crowdsourcing, you are dealing with people. With that said, specically on our platform, we take this matter extraordinarily seriously. In fact, an upcoming eBook from TopCoder solely focuses on this topic and it canvasses 8 questions on intellectual property in Open Innovation that you should be asking, and why they matter to you.

As we just mentioned, an upcoming eBook will focus on this issue entirely, but here are some main points to understand right now. Various levels of NDAs On our platform, you have the ability to require as many levels of NDAs as you feel necessary. We have provided for clients, smaller select pools of participants sometimes based on country origin or other factors that matter to the client, and on occasion we have then asked these individuals to sign more restrictive NDAs directly with the client. We oer this exibility to our clients so that they can reach their desired level of comfort with any given project. However, we guide our clients to be as open as possible as often as possible as to not restrict participation from any community members who are looking to compete. The Role of Atomization In the previous burning question we dened atomization, so chalk it up to good timing because now well share how atomization positively impacts your work from an intellectual property sharing standpoint.
PAGE31
As simply as this can be stated; when the work is broken down into its smallest pieces of the build and launched as separate competitions, this process innately helps to protect the whole or entirety of the project from being unveiled. Often, we have clients that simply cannot discuss an innovative project publicly due to the competitive advantage they can potentially gain, so they opt to not share with our community the entire project. They may still desire user interface innovation, or an algorithmic solution that speeds up a computational engine, but they simply dont feel comfortable unveiling the entire project to our global community. When you need to, on-demand, you can highly isolate work, have the isolated work created via TopCoder while your internal team works on pieces of the project that you perhaps deemed too sensitive to be shared externally. Again, this is about your choice and your gained exibility via Open Innovation on the TopCoder Platform. As we stated earlier, an upcoming eBook from TopCoder focuses solely on the challenges of IP in Open Innovation. Subscribe to the TopCoder Blog at www.topcoder.com/blog/ (upper right corner of page) and you wont miss this eBooks release.
PAGE32
BURNING
QUESTION #3
HOW CAN WE TRULY SCALE OPEN INNOVATION?
?
?
?
?
?
?
?
At the risk of entering broken record territory we will say the following: Open Innovation is about the transformation of how you approach productivity and continuous value creation. If it were instead about hosting a singular one o challenge that you were never likely to repeat (from a process stand-point) that might be better classied as a complicated marketing initiative you were attempting in order to get new eyes around your brand, with the possible consequence of some gained innovative perception. You may even have outstanding and innovative results from that one o competition. But that isnt repeatable, that isnt scale, and therefore that isnt transformative. From the standpoint of the community or crowd you are working with, when you are consistently launching, eectively managing, and executing contests through the delivery of the newly created asset, the individual community members take notice. If they see a robust pipeline of work stemming from you and your company, paired with your professional and consistent approach to Open Innovation management, they will continually register for and compete on your work. Having that type of community following can pay big dividends and help you to achieve scale even faster.
PAGE33
HOW CAN WE TRULY SCALE OPEN INNOVATION?
?
?
ANSWER:
? ?
There are 4 key factors you should look for in ascertaining whether a specic Crowdsourcing or Open Innovation platform can help you achieve scale. They are: Repeatability How easily can you Lather. Rinse. Repeat.? How often can you launch open competitions and is the eort to do so easy or hard? Delve deep into the process of launching, managing, and closing out individual contests. If it seems dicult to operate one competition at a time, you need to ask yourself how you plan on achieving the type of scale you truly want to arrive at. The solution, community, or platform you choose to use should allow you to create in a massively parallel fashion. Predictability For any given competition, contest, or challenge, you should have a legitimate understanding of the % of success you are likely to achieve. Of course, no prediction is perfect, but fundamentally, understanding how likely it is that you succeed (and understanding the true cost of time and money spent if your competition fails) plays a formidable role in how often you attempt. Open Innovation (and to a lesser extent Crowdsourcing) is at their core, methodologies that lower the level of risk to a specic threshold that allows for more creative attempts endeavored. Ask the very dicult question of how predictability is measured on any platform you are considering. The answer will likely reveal how mature the platform is, or is not.
PAGE34
Dierentiation If you can repeat only one type of task, contest or challenge even if you can routinely predict a favorable outcome it likely limits how valuable that community or crowd can be for you. If you are contracting a Crowdsourcing localization service to only handle your document translation, which may save you money and speed up your translation processes, but as we discussed earlier; this does not make you more innovative, it simply makes you more ecient. Your ability to canvass a wide variety of tasks and challenges is an important aspect to how you will ultimately scale. The more types of specic work you can accomplish and innovation you can attempt, the greater the transformation that can take hold. Results Open Innovation and Crowdsourcing models are mainly outcomes based, meaning you are paying for best in breed results (the winning solutions) instead of paying for eort. The level of results you can receive is directly correlated to the level of talent within any given community or crowd. This is as simple as doing your homework. Understand what this community or crowd has accomplished, what it likely can accomplish, and for extra credit ask how the platform would adjust to support a newer technology or task it has yet to encounter. The past results of a community and how they likely will perform for you going forward ties directly to your potential success. If you are condent on a particular platform, human nature would dictate you will likely scale that platform to maximize the outcomes.

Lets quickly run through each of the 4 key factors impacting your ability to scale Open Innovation and Crowdsourcing starting with repeatability. Repeatability With thousands and thousands of specialized contests specied, prepared, launched and successfully delivered, TopCoder has the most experience the world over in repeatable Open Innovation success. Pair this experience with Copilots, an enterprise ready management tool (Cockpit), and robust analytics & reporting and you are delivered the worlds most comprehensive productivity platform the moment you become a client. With TopCoder, you can Lather. Rinse. Repeat. on-demand, as many times as you would like. Predictability Lets start with a stat. TopCoder has a contest fulllment rate over 90%. This means, better than 9 out of every 10 contests are successful, and that means; delivered on time, at the predened budget, at a quality level all parties felt was strong. Lets follow that up with a point. For the less than 9% of competitions that fail, you dont pay. Youve heard the mantra: Fail fast, fail often, fail cheap. We procure that type of failure, which helps ensure your success.
PAGE35
Technically speaking: Almost every type of competition on TopCoder is tied to a predictive engine that takes into account the level of specialized talent registering to compete on your work, their predictability rating (measuring how often an individual submits a solution after deciding to register for the given contest), and other factors. Before a contest even enters the competitive phases you have a clear understanding of the likelihood of that specic contests success. Dierentiation We discussed earlier TopCoders 3 Pillars of Digital Creation, spanning idea and design contests, to software construction, all the way over to Big Data algorithmic challenges. There is a very large variety of work you can accomplish and innovation you can attempt on the TopCoder Platform. The variety of work you, your team, and your enterprise decide to make happen is just that, your choice. You can master it all and produce like never before. Results As mentioned earlier, results are directly correlated to the talent you are accessing. The TopCoder community is now over 485,000 members strong and in 2012 alone we hosted over 4,500 separate competitions. With a fulllment rate above 90%, our clients enjoy an outcomes based model that is focused on results and their innovative success.
PAGE36
YOUR NEXT MOVE:
BEGINNING YOUR OPEN INNOVATION AND CROWDSOURCING JOURNEY

Why did you download this starting guide and read through what was a good deal of content? Perhaps it was simply to learn about newer methodologies and lexicons spurred on by your inner curiosity to continuously improve yourself. We applaud that, sincerely. But likely your want was deeper. Perhaps as an enterprise leader, (or an aspiring one) you innately feel the change that is happening all around you. This starting guide and your journey beyond reading it were always about your personal transformation and your ability to dictate your own innovative path forward. The reality of the situation is simple. If you dont master the methodologies that breed Open Innovation and Crowdsourcing success, your competitor (or soon to be competitor) will. And though we appreciate you downloading and consuming this starting guide, and though it may be the catalyst for your next action, it is not a substitute for it. Your next move in Open Innovation and Crowdsourcing is to get involved in some capacity. There are communities to join that focus on an interest you hold dear, co-creation opportunities to compete on where a specic skill of yours will serve you well, and of course, there are competitions to launch and manage to a successful result. Any of these beginnings gets you closer to your mastery of Open Innovation, and the result of your mastery is highly predictable. You will succeed like never before.
Five frogs sit on a log. Four decide to jump in. How many frogs are left on the log? Five. Deciding is not doing.
Welcome, jump in!
PAGE37
?
?
?
?
?
?
Brought to you by
10 Bur ni n g Qu e s t i o n s o n C ro w dsourci ng:
YOUR STARTING GUIDE TO OPEN INNOVATION AND CROWDSOURCING SUCCESS.

TopCoder Crowdsourcing

Transféré par

Informations du document

Description originale:

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

TopCoder Crowdsourcing

Transféré par

Droits d'auteur :

Formats disponibles

?

10 BURNING QUESTIONS ON CROWDSOURCING:

YOUR STARTING GUIDE TO OPEN INNOVATION AND CROWDSOURCING SUCCESS.

10 BURNING QUESTIONS ON CROWDSOURCING

WELCOME, JUMP RIGHT IN!

10 BURNING QUESTIONS ON CROWDSOURCING

WHAT YOU ARE ABOUT TO ENJOY

THE STARTING GUIDE BREAKDOWN

GROWTH HOW DO I SCALE OPEN INNOVATION AND TRANSFORM YOUR ENTERPRISE?

10 BURNING QUESTIONS ON CROWDSOURCING

PART 1: LOGISTICS - HOW DOES CROWDSOURCING

PART 2: EXECUTION - HOW DO I PRODUCE IN

PART 3: GROWTH HOW DO I SCALE

10 BURNING QUESTIONS ON CROWDSOURCING

Lets dive into the questions.

10 BURNING QUESTIONS ON CROWDSOURCING

Why this question matters to you:

10 BURNING QUESTIONS ON CROWDSOURCING

Specically on the TopCoder EOI Platform:

10 BURNING QUESTIONS ON CROWDSOURCING

Why this question matters to you?

10 BURNING QUESTIONS ON CROWDSOURCING

Specically on the TopCoder EOI Platform:

Do TopCoder competitions sometimes need an extension?

10 BURNING QUESTIONS ON CROWDSOURCING

Why this question matters to you?

10 BURNING QUESTIONS ON CROWDSOURCING

Specically on the TopCoder EOI Platform:

Selecting Winners on Each Track

SUBJECTIVE (Client Decides Winner)

OBJECTIVE (Rigorous Scorecards Decide Winner)

PERFORMANCE (Best Solution against pre-determined metrics wins)

10 BURNING QUESTIONS ON CROWDSOURCING

LET'S DEFINE IT WHO ARE PEER REVIEWERS AT TOPCODER?

10 BURNING QUESTIONS ON CROWDSOURCING

PART II: EXECUTION HOW DO I PRODUCE IN OPEN INNOVATION AND CROWDSOURCING?

10 BURNING QUESTIONS ON CROWDSOURCING

PART II: EXECUTION HOW DO I PRODUCE IN OPEN INNOVATION AND CROWDSOURCING?

Why this question matters to you:

10 BURNING QUESTIONS ON CROWDSOURCING

PART II: EXECUTION HOW DO I PRODUCE IN OPEN INNOVATION AND CROWDSOURCING?

Specically on the TopCoder EOI Platform:

10 BURNING QUESTIONS ON CROWDSOURCING

PART II: EXECUTION HOW DO I PRODUCE IN OPEN INNOVATION AND CROWDSOURCING?

Why this question matters to you:

10 BURNING QUESTIONS ON CROWDSOURCING

PART II: EXECUTION HOW DO I PRODUCE IN OPEN INNOVATION AND CROWDSOURCING?

Specically on the TopCoder EOI Platform:

LET'S DEFINE IT WHAT IS NEAR-FIELD REPURPOSING?

10 BURNING QUESTIONS ON CROWDSOURCING

PART II: EXECUTION HOW DO I PRODUCE IN OPEN INNOVATION AND CROWDSOURCING?

Why this question matters to you:

10 BURNING QUESTIONS ON CROWDSOURCING

PART II: EXECUTION HOW DO I PRODUCE IN OPEN INNOVATION AND CROWDSOURCING?

10 BURNING QUESTIONS ON CROWDSOURCING

PART II: EXECUTION HOW DO I PRODUCE IN OPEN INNOVATION AND CROWDSOURCING?

Specically on the TopCoder EOI Platform:

LET'S DEFINE IT WHAT IS COLLABORATIVE QUESTIONING?

10 BURNING QUESTIONS ON CROWDSOURCING

PART II: EXECUTION HOW DO I PRODUCE IN OPEN INNOVATION AND CROWDSOURCING?

Why this question matters to you:

10 BURNING QUESTIONS ON CROWDSOURCING

PART II: EXECUTION HOW DO I PRODUCE IN OPEN INNOVATION AND CROWDSOURCING?