(1) Which of the three company's approaches to using people analytics for talent acquisition and development is most appealing (or most concerning)?

(2) Should Fukuhara turn on the most advanced part of the artificial intelligence engine, allowing GROW not just to provide recommendations to clients about whom they should hire, but also (based on performance and attribute data of previous hires) to overrule clients' specifications (or biases) about the competencies they should be targeting in their ideal hires?

sorry about that

For the exclusive use of Y. Sherhan, 2018. GROWLwing Anicial Intelligence to Sumun intellige summer 2017, secured a $6M Series A round with capital from the University of Tokyo Edge Capital, the Tokyo University of Science Investment Management Company, and the Keio Innovation Initiative. These investments made IS the first and only venture in Japan funded exclusively by university VC which Fukuhara viewed as a testament to GROW's dual purpose for students and companies After closing the Series A financing, IGS focused on client acquisition by aggressively targeting HR decision-makers at prominent firms and demonstrating IS's tehnical capacity to do complex data analyses en companies human resources processes. More client contracts with major Japanese firms and a medio-friendly story (ob hunting frustration could be added to death and taxes as unpleasant certainties in Japanese society) helped GROW increase its users from 2.000 in December 2016 to 74,000 by June 2017. a small but meaningful share of Japan's annual university graduates (roughly 650,000). How GROW Works Once a ser created a GROW account and completed an in-app tutorial (Exhibit 1), she or he could evaluate the competencies of another user (classmate, co-worker, acquaintance, etc.), complete a self- evaluation, or complete the LAT. IGS used its Al algorithms to analyze the resulting data from both candidates and evaluators, in order to develop and customize HR-related services for clients and users. Competencies To power GROW's Al engine (see Exhibit 4 for an overview of how it worked), ICS first developed a list of competencies and associated queries that met several specific criteria (Exhibit 2). Given the many ways a competency could be defined, each competency was initially approached through six or seven questions (called queries). However, to increase the chances that evaluators would complete the assessment IGS then used principal component analysis to reduce the number of queries down to the most relevant and least redundant there Table 1 Competencies are defined through three queries Competency Creativity Query 1: Shrdes not merely imitate others when doing a task she tries to add her/his on value Query 2: She is good at coming up with ideas no one else has thought of Query 3 She can combine existing ideas to create something new. Evaluators were asked to answer cach query through a four-level rubric, preventing a neutral response. Generally, the four lovels indicated whether the candidate (1) rarely, (2) sometimes, (3) frequently, or (4) nearly always exhibited the actions or traits of each query (Exhibit 2), but the rubrics also contained specific details to help evaluators accurately distinguish among the levels (Table 2). Table 2 Competencies are evaluated through a four-level rubric Competency Creatioity faith regard to Query 1 above) Rubric 1: Se merely amatates others when doing a task Rubric 2: Once in a while. She will do something different Rubric 3: She has the right attitude to do things in a way that adds her/itis oron bulue. Rubric 4: By doing things in her/his can asy, Vine actually adds her/uts on babe For the exclusive use of Y. Sherhan, 2018. GROW.Uning Alicial Intelligere Se Human Intelligen inemased us need for highly capable managerial talent. Interested in increasing its pool of such candidaters, MChegan discussions with ICS to use GROW. To assist MC in sourcing new candidates for its applicant pool. I first worked with MC to create within GROW an algorithmic model for the deal" MC employee. IGS and MC then tan the algorithm on candidates alsady registered in MC's application database who were also among the GROW users who had completed GROW's competency and disposition screening Based on a hypothesis that pers of "ideal candidates might also be ideal" (hirds of a father flock together), IGS created a social graph of the ideal candidates, identifying those individuals who were among the five or more peers who had given the ideal candidates feedback (Figure 2), 165 then passed this information along to MC which used the social graph to identify those peers who had not registered in MCs hiring database MC decided to these peers of "ideal candidates to an information session about MC Fifty potential candidates attended this session. Interestingly, many of them already had job offers from top companies such as Google and Goldman Sachs. Yet, by speaking with current employees and getting to know more about MC.. number of them not only applied to MC after the session, but eventually decided to join MC after passing the interviews To MC, GROW's greatest value was as a tool to help discover talent who had not discovered them Figure 2 Using the Social Graph of Mitsubishi Corporation's (MC) Candidates Det Anal medel "Meat employee 1.Sentite model both MC and pool och Anavedenih lowed these por of the balls to ident the do GROW valors 22 & GROWing Forward With more client data on how GROW's commended hires were performing, and more applicant data (IAT and peer competency rating) from an increasing number of users. GROW's Al was poised to deliver ever more robust hiring recommendations to clients. But as Pukuhara's Albecame smarter, what about his human clients? Was it time to educate them on the best ways to use GROW? If so, which of the ways in which his clients were using GROW were likely to be better and less potentially miskin? Fukuhara-san also cautiously saw potential to expand GROW's capabilities. As ICS collected data not just on current recruits but also on the performance of previously recommended his GROW'S AI had the potential to improve not just the hiring process but also the hiring criteria. Was it time to tum on the reinforcement machine learning" Al (Exhibit 4) such that the Al would be able to overrule Some of the client-defined characteristics of an ideal candidate that had not worked well in the past based on how hired individuals ultimately performed at work? In short could Pukulana's Al become smarter than his clients -and how would his clients feel about such a proposition and the ending of control it would require Fukuhara wondered in what ways GROW should grow, dowe Operation (MGT3121 by Gery LGUNY Collegato ary 2011 2018 For the exclusive use of Y. Sherhan, 2018. For the exclusive use of Y. Sherhan, 2018. GROW Ling Artificial Intelligence to Souman Intelligence Exhibit 1 Using Grow 1. Download App OS or Android 2. Complete Basic Protile Information 3 Complete Academic Protile Upload Resume GR w 4. Find friends already on GROW 5 Invite trends not on GROW to join 6. Make or respond to friends feedbacktes uests Source 10 cury documents Marga OT 3121 by Corynow, CUNY Baruch College from January For the exclusive use of Y. Sherhan, 2018. GROW, Ling Artificial Intelligence to fireen Human Intelligence Weighting Users received ratings from multiple evaluators (on average four to five in addition to the self- evaluation, but each evaluation was not weighted equally. Instead. GROW employed a patent- pending probabilistic (Bayesian) machine learning (AI) algorithm that used data of how that competency had been evaluated historically (called priors) along with many data points about an evaluator to determine the posterior likelihood that such an evaluation was genuine and calibrated to reality (Exhibit 4). For example, every query-rubric pair had its own data regarding the amount of time it historically took users to complete that evaluation (from reading the query and rubrics to making the sessment). So, if an evaluator answered much faster than most, that might lead the algorithm to decrease the weight of his evaluation. Similarly, if an evaluator gave more diverse ratings over time- some I'ls and 4's-she might be deemed more discerning and therefore much more reliable than an evaluator who gave all 3's The algorithm also took into account an evaluator's IAT results, as certain personality tendencies (such as conscientiousness) often led to more reliable evaluations. Another key data point was the social graph of the evaluator, an evaluator that gave evaluations for multiple users in various social networks was typically more reliable than an evaluator who only joined the app to give evaluations to one particular sr. Personality Traits The Implicit Association Task was a well-known test in social psychology to reveal attributes and biases people have and the test has been connected to personality traits. In GROW, users progressed through a series of implicit association tasks, in which they drag certain attributes, appearing at the hottom of the screen to the containing the correct attribute(for example, talkative to extroverted) (Exhibit 3). The box containing the correct attribute was either juxtaposed or matched with a concept (mysell, others, and the manner in which the user swiped the moveable attribute was assessed to predict the personality traits of the user. GROW's patented mobile IAT utilized unsupervised machine learning to reveal anomalies in patterns of swiping behavior and better predict personality How Organizations Use GROW After numerous rounds of testing ICs was ready to provide GROW as a software-as-a-service (SAAS) tool for interested firms. There was overwhelming interest. With the advent of online resume submission, digital outreach and apps specifically designed to help students apply effortlessly to many potential employers, applications were skyrocketing the average undergraduate in Japan was sending between 50-100 resumes, or "Entry Sheets" (ES), to companies during job hunting season. But even as companies were flooded with applications, their approach to processing those applications had not changed much they still relied heavily on laborious resume screening and several time-consuming rounds face-to-face interviews. As a result, they simply could not keep up and the quality of their new hires were suffering as result. GROW presented a very attractive solution, although as firms adopted it, it became clear that firms were using the tool in different interesting and even unexpected ways. Septeni Holdings Septeni Holdings (Septeni) was founded in 1990 to develop a human resource consulting business at the height of bubble-era Japan. After the bubble burst Septeni shifted its focus to internet advertising just as the internet was taking off. In 2017, Septeni was one of the largest internet marketing companies in Japan, and it catalyzed growth by adopting unique approaches to recruit and cultivate the best entrepreneurial talent who could then effectively grow subsidiaries and therefore revenues. For the exclusive use of Y. Sherhan, 2018. GHOW. Ling Algemee to the Human A Short History of IGS Prior to starting IS. Fukuhara was a managing director at awt management firm Barclays Global Investors (BGT) where he made investment decisions huwd on computer-driven models (se Exhibit for Pukuhara's work history. The mantra at was "quantify everything and it instilled kuhara with the belief that quantitative judgment would result in greater risk-adjusted retums. As Fukuhara moved up in the organization, he spent time managing the data and more time managing people, and he began to ponder why not qualify personal capabilities as well? Like investors making at management decisions, he noted that people can fail to othvely and perhaps quantitatively) assess themselves and others. Could the advantages of quant Investing applied to evaluating and developing people Following Black Rock's acquisition of HGL and sensing an opportunity to pursue his curiosities for himsel Fukuhana left the firm and founded an educational venture, ICS At first. Pukuhana Tan IGS as a small private cram school in a bustling part of Tokyo. Doing so provided him with a chance to observe firsthand how young people grow, how they respond to feedback and how they learn to their own skills. An avid reader of philosophy educated in both Japan and abroad. Pukuhara designed a novel currkulum of philosophy, critkal thinking, debate, and English as a second language (ESL) which drew interest not just from parents and students in Japan but also from schools and governments to whom Pukuhara consulted. Through the conversations and thinking back to his own experiences with 360 degree feedback at INSEAD and BGI, Fukuhata felt increasingly confident that people's behaviors could change for the better through more frequent foedhack-once or twice a month rather than once or twice a yeart Yet, drawing on his Japanese roots he knew changing honest feedback was uncomfortable for many Japanese, making it an uncommon event. To Fukuhara, the hiring process was a med opportunity to collect utiline, and exchange such fedhack in a systematic way and at a juncture when individuals were likely to act on it Once again Fukuhara saw a connection to his former work. At BGI during his years during his years as an FX and global fined income researcher, he developed early deep learning and other advanced statistical methods to support the quant investment platform. Pukuhara sought to build a similarly scalable technology for feedback in the hiring process. In early 2015, he began holding weekly hrainstormingusions with former colleagues, friends, and researchers to flesh out a tool that would both help students understand their strengths and weaknesses and assist HR in hiring, As a nod toward the desire to help students grow, the tool was named GROW. With marketing support from one of Japan's most prominent media companies. Asahi Shimhun, as well as funds earned by spinning off IGS s education arm, Fukuhara turned to focusing ICS on developing the two-sided GROW platform. GROW Through focus groups with student users and countless meetings with executives at various firms and institutions. IGSs agile team developed GROW as a gamified, mobile consumer app that allowed students to gift cach other feedback on various competencies and discover their personality traits through a modified IAT task. Needing companies to join the platform. IGS added a B2B2C strategy and began providing GROW to HR functions in organizations. Farly clients included mostly tech/manufacturing companies like DNA and Calsonic Kansel, though clients used the service in various ways to identify star players in their firms, to assess those who had already received job offers to join their firms (potentially to place them in certain roles), or to ascertain which subset of the 25 competencies were strategically critical yet consistently underrepresented in employees' test results. In need of more cash to improve the technology, increase client acquisition, and expand the user base (grassroots efforts signed up just a few hundred), Pukuhara set off for meetings with VCs and, by 2 Theodore ay by Yucal Bhetani servo Opension Management IMOT 12) by Cory Labnow. Car-Bartholgeboy 2018 For the exclusive use of Y. Sherhan, 2018. GHOW.Using Artificial telligence to Score Human Intellige 415020 Exhibit3 Implicit Association Test (IAT). GROW's personality assessment The below are based on the Big Five personality traits, developed independently by several groups of researchers in the 20 century, Extroverted You are proactive, dynamic, action-oriented, and always looking for external stimulation through interaction. On the other hand, your actions can be 1 cable Introverted You are reserved and prefer a stable environment. You do not look for external stimulation and find stability within yourself. On the other hand, you can be a cowand can be slow in taking actions. Open You try to obtain new information and expand your knowledge base and 2 experience. On the other hand, you can be unrealistic and can have idealistic ideas Conservative You are content with the current situation and grounded. On the other hand, you do not like changes and tend to stick with the authority and traditions Sensitive You can be sensitive to risk and tend to avoid risk by being prudent. On the other fund, you can get really nervous and emotionally unstable when placed in a 3 Mressful environment You are not influenced by risk and are always emotionally stable. On the other lund, you are insensitive to other peoples' feelings or environment. Cooperative You are a wood listener can understand others, and create cooperative relationships. On the other hand, you tend to think of others too much and tend to be a follower. Therefore, the cooperative person can lack leadership and can tend 4 to be a shrinking violet within the group Independent You can have a strong character, have the ability to generate original and new ides, and lead a group. On the other hand, you have a tendency to be selfish and too competitive You have a strong will and make efforts to achieve a goal every day. On the other 5 hand, you can be a perfectionist and can frustrate yourself and others. Laidback You accept who you are and the environment and are flexible. On the other hand. you can lose your way and can make careless judments In an example of GROW's customised, mobile friendly test concepts of self and others (in orange) are randomly paired with personality traits (in yellow). The user is asked to swipe the moveable yellow box (labeled Tallarive in this example) at the bottom of the screen to the correct attribute (extroverted). Someone who considers themselves extroverted may besitate when associating others with being more talkative. This "hesitation is tracked and analyred in two ways in the pattern of the swipe (straight to the left or in a curved fashion) and in the time tracker (at the bottom of the screen). Established priors can then help determine the posterior likelihood of user's personality type. Source Scompany documents 10 hy by the nerve Operation ManagemMT 3121) het by Cory Larow, CUNY- Baruch Cobagtromy For the exclusive use of Y. Sherhan, 2018. 15.00 GROW. Laing Antintelliger te Screen Human Intelligence To che talent. Septeni had traditionally followed the common Japanese approach of inviting university students to its offices in Tokyo to take part in multiple rounds of group interviews, in which groups of applicants would be monitored as they completed teamwork-based tasks that simulate typical work at Septeni. But that approach had significant drawbacks for Septeni, including the need for students to visit Sepleni in person (which largely excludes students living outside of the Tokyo anual the need for students to be sufficiently aware of and interested in Septent as a potential employer to invest the time and money to join the group interview days (which, as a mid-sized company, was not always the case), and the need for students to perform exceptionally well in the one-shot group interviews, as that was the only chance Septeni had to assess them before making a hiring decision To address the first issue Septeni had created a new online hiring process, composed of one video Interview plus a web based inquiry, for students living outside of the Tokyo area who could therefore opt out of paying expensive travel costs for the interview. That, however, left Septeni without the data it had traditionally collected through the in-person group interviews. Septeni turned to GROW as a substitute for that data. By collecting evaluations from peers throughout a student's time in university, GROW fit well with Septeni's internal evaluation criteria, which was focused on personality traits and on capabilities an individual demonstrated as she or he worked with others over time. In 2016, Septeni provided ICS with data on prior year candidates and interview outcomes to help ICS "train the Al algorithm, from which is developed a supervised machine learning algorithm to accurately predict which candidates--past and future-would pass Septeni's group interviews, Early results with GROW in the 2017 recruiting season were promising. Not only did GROW's predictions dosely match the outcomes of Septeni's internal evaluations, but it doubled the talent Septeni sourced from outside Tokyo, made potentially obsolete the group interview process, and increased Septeni's name recognition among students (as GROW's other clients included well-known firms in Japan). The final outcome a 90% reduction of the overall process effort whilo Sepleni's year-over-year acceptance rate of its job offers jumped four-fold, all with no apparent impact to candidate quality. All Nippon Airways Japan's largest airline, All Nippon Airways (ANA), was consistently rated among the most popular companies in the eyes of students seeking jobs after graduation. To build its pipeline of future senior leaders, ANA sought to identify promising students by screening the tremendous number of applications they received every year. With only a limited number of HR staff, however, ANA feared missing needles in the haystack students with the potential to be future leaders but who were screened out too early in the recruiting process First IGS worked with ANA to prioritive ten competencies it would highly value in its new recruits. Students interested in ANA then used the GROW app have their competencies and personality traits assessed, which was used to create a "total score." Based on all of the other data IGS collected about a student and her evaluators, GROW's Al engine also produced a "confidence score" to rate the degree of confidence IGS had in that total score. ANA then plotted each applicant on a single graph, with "total score" on the x-axis, confidence score" on the y-axis, and the color of the dot representing how far into the screen process the candidate progressed (application received, invited to first round, invited to second round, invited to third round, finalist, received job offer) (Figure 1). In lapar, it was common for companies to hold interview days at their leadquarters and expect all interested candidates to show up in person at their own expense IMT 2121) by Cory Labow. CUNY- Baruch College women For the exclusive use of Y. Sherhan, 2018. GROW, Ling Ancillatelligence to SHuman Intelligence Exhibit 2 Competency Evaluation in GROW Tech.competency was selected from hundreds of competencies and skills reported in the sociales literature on the basis of veral criteria: 1) representation across different countries and culture 21 correlation to success in relevant job functions as verified by cutive search forms and 3) essable by own. The winnowing down of competencies was done by Dr. Mitsuru Kimura of the University of Tokyo An excerpt of each competency, translated from Japanese, follows with Tested BE When a mere complete the duckman web permend en hew the improve 86 . 3 54 comun dement dyty ShanDongTay Cory Labo CNY - Colection 2018 For the exclusive use of Y. GROW:Ling Aftatellite for Human Intellige Figure 1 ANA can use GROW to interview promising candidates it would have seened out Comic Son Total Score SANA .company docum The data realed several insights 1. ANA could use GROW to screen out candidates that were unlikely to make it to a final interview, as candidates with a less than 4 confidence score and less than 5 total score were extremely unlikely to become a "green dot": 2. The chustering of green dots in the upper right of the graph font confidence to the use of GROW as a tool for ANA'S HR staff and 3. There were many students who did not make it past the application screening and yet could not he distinguished from final round interviewees using GROW. To investigate the third insight further. ANA invited around 423 students who had high GROW scores to interview along with the candidates who passed their traditional process. To ANA's surprise, some of the GROW students invited received perfect scores in the interviews, leading HR to realize that by using GROW data in a supervised machine learning manner, GROW could help surface promising candidates ANA would have missed otherwise As a result, rather than focusing on screening students out ANA decided to use GROW to screen students in, further developing and fine-tuning its inclusion criteria. With these parameters, ANA could use GROW to more accurately target clusters of students with high potential to advance through their recruitinent process Mitsubishi Corporation One of Japan's largest and most storied enterprises, Mitsubishi Corporation (MC) operated businesses in diverse industries, including industrial finance, metals, machinery, chemicals, new energy, infrastructure, finance, technology, daily living essentials, and more. In 2017, MC's activities extended far beyond its traditional trading model to include active participation in the management of its businesses. While the firm historically had few issues attracting Japan's best and brightest graduates (it was often number one in popularity among new graduates), the evolution of its business model had This documents and only by the Brio Operations Management (MGT31 taught by Cory Lahanow, CUNY.Baruch Chege tromy 2018 For the exclusive use of Y. Sherhan, 2018. HARVARD BUSINESS SCHOOL 9-418-020 STRAN FERNSTEIN PAUL MCKINNON PAUL YARAR GROW: Using Artificial Intelligence to Screen Human Intelligence Masahiro Pukuhura, founder and CEO of Tokyo-tused people analytics startup Institution for a Global Society (CS), took a break from poring over data to question how it would be the used. In the seven years since Fukuhara had founded is, its solution to evaluate job candidates - "GROW-had grown quickly, and the wide variety of ways clients used it simultaneously excited and concerned him GROW. an iOS and Android app (Exhibit 1), consisted of two proprietary components a competency assessment and a personality assessment. To assess competencies GROW employed a peer-feedback tool to reveal 25 specific competencies which ICS had chosen based on extensive social science research (Exhibit 2). To assess personality, GROW employed a gamified version of the Implicit Association Test (IAT), an established assessment of hidden bias in social psychology, that individuals could play on mobile devices (Exhibit 3). While neither component was particularly novel, what GROW did with them was it used artificial intelligence (AI) learning algorithms to analyze every speck of assessment data from both the candidates and the evaluators, looking for patterns to improve its ability to accurately screen candidates over time (Exhibit 4). In place of human "intuition," GROW used "big-data"-disparate data points across many people to develop a scientific, objective, and constantly improving engine to recruit screen and develop human capital. So far, it seemed to work. For example, in a test of GROW, one client had both its HR professionals and GROW evaluate the same 200 students. GROW not only surfaced nearly the same top 50 candidates (the two lists of 50 were statistically indifferentiable), but more importantly it did so with specific data-based competency-based justifications. As of June 2017, GROW had 74,000 users, including students at both prestigious and lesser-known universities. Clients included Mitsubishi Corporation, All-Nippon Airways (ANA), Septeni, DeNA, Rakuten, AXA, and many others. Even government entities like Japan's Ministry of Economy, Trade ke Industry (METI) and the UAE were getting involved. The widespread interest in GROW was both an opportunity and a challenge. On the one hand, it provided Pukuhata with a growing base of users, data, and institutional support. On the other hand, Fukuhara wondered if he should play a stronger role in strategically focusing the use of GROW where it was likely to have the most meaningful (and least potentially misleading) impact. Plemeniele Pail Menon PYMA perpow whence of Nobuo 12 Ni Tarand Almosthenewed and approprio by company de Funding for the deleted provided by Harvard School and by the company are developed Cormichon Follows of Harvard College Tonder weer que presion to produer material call 1-800-2005 wird scholing MA pelowww.libeparvard.edu. This publication may the dig.photocopied www produce posuded, without the permission of Harvard School ta multorte brune cry by Semannsenice Operations Management (MGT 3121) taught by Cory Labanow. CUNY - Baruch College from January 2012 2018 For the exclusive use of Y. Sherhan, 2018. 411.00 GROW. Using Artificial Intelligence to See Human Intelligence Exhibit 4 How the Artificial Intelligence in GROW Works 4+ Emites Par En User Dette Captured buth GROW App Competency Evaluator Ratings of User Personality IAT User Personality IAT Umupervised Machine Learning Al pof the Supervised Machine Learning Al ww Multiple Per Rating of User Candidate Recommendation Bry 3 L! - Duta Soome Reinforcement Machine Learning AI Be -GROW Date dom - Company Data Current) Artificial Intelligence (AD) a) Aitical Intelligence AD Come Recommendati Hiring Company) cally Sour Crewritt 11 Operations Management MOT 5121) taught by Cory Labanow. CUNY Baruch College tomayo For the exclusive use of Y. Sherhan, 2018. GROWLwing Anicial Intelligence to Sumun intellige summer 2017, secured a $6M Series A round with capital from the University of Tokyo Edge Capital, the Tokyo University of Science Investment Management Company, and the Keio Innovation Initiative. These investments made IS the first and only venture in Japan funded exclusively by university VC which Fukuhara viewed as a testament to GROW's dual purpose for students and companies After closing the Series A financing, IGS focused on client acquisition by aggressively targeting HR decision-makers at prominent firms and demonstrating IS's tehnical capacity to do complex data analyses en companies human resources processes. More client contracts with major Japanese firms and a medio-friendly story (ob hunting frustration could be added to death and taxes as unpleasant certainties in Japanese society) helped GROW increase its users from 2.000 in December 2016 to 74,000 by June 2017. a small but meaningful share of Japan's annual university graduates (roughly 650,000). How GROW Works Once a ser created a GROW account and completed an in-app tutorial (Exhibit 1), she or he could evaluate the competencies of another user (classmate, co-worker, acquaintance, etc.), complete a self- evaluation, or complete the LAT. IGS used its Al algorithms to analyze the resulting data from both candidates and evaluators, in order to develop and customize HR-related services for clients and users. Competencies To power GROW's Al engine (see Exhibit 4 for an overview of how it worked), ICS first developed a list of competencies and associated queries that met several specific criteria (Exhibit 2). Given the many ways a competency could be defined, each competency was initially approached through six or seven questions (called queries). However, to increase the chances that evaluators would complete the assessment IGS then used principal component analysis to reduce the number of queries down to the most relevant and least redundant there Table 1 Competencies are defined through three queries Competency Creativity Query 1: Shrdes not merely imitate others when doing a task she tries to add her/his on value Query 2: She is good at coming up with ideas no one else has thought of Query 3 She can combine existing ideas to create something new. Evaluators were asked to answer cach query through a four-level rubric, preventing a neutral response. Generally, the four lovels indicated whether the candidate (1) rarely, (2) sometimes, (3) frequently, or (4) nearly always exhibited the actions or traits of each query (Exhibit 2), but the rubrics also contained specific details to help evaluators accurately distinguish among the levels (Table 2). Table 2 Competencies are evaluated through a four-level rubric Competency Creatioity faith regard to Query 1 above) Rubric 1: Se merely amatates others when doing a task Rubric 2: Once in a while. She will do something different Rubric 3: She has the right attitude to do things in a way that adds her/itis oron bulue. Rubric 4: By doing things in her/his can asy, Vine actually adds her/uts on babe For the exclusive use of Y. Sherhan, 2018. GROW.Uning Alicial Intelligere Se Human Intelligen inemased us need for highly capable managerial talent. Interested in increasing its pool of such candidaters, MChegan discussions with ICS to use GROW. To assist MC in sourcing new candidates for its applicant pool. I first worked with MC to create within GROW an algorithmic model for the deal" MC employee. IGS and MC then tan the algorithm on candidates alsady registered in MC's application database who were also among the GROW users who had completed GROW's competency and disposition screening Based on a hypothesis that pers of "ideal candidates might also be ideal" (hirds of a father flock together), IGS created a social graph of the ideal candidates, identifying those individuals who were among the five or more peers who had given the ideal candidates feedback (Figure 2), 165 then passed this information along to MC which used the social graph to identify those peers who had not registered in MCs hiring database MC decided to these peers of "ideal candidates to an information session about MC Fifty potential candidates attended this session. Interestingly, many of them already had job offers from top companies such as Google and Goldman Sachs. Yet, by speaking with current employees and getting to know more about MC.. number of them not only applied to MC after the session, but eventually decided to join MC after passing the interviews To MC, GROW's greatest value was as a tool to help discover talent who had not discovered them Figure 2 Using the Social Graph of Mitsubishi Corporation's (MC) Candidates Det Anal medel "Meat employee 1.Sentite model both MC and pool och Anavedenih lowed these por of the balls to ident the do GROW valors 22 & GROWing Forward With more client data on how GROW's commended hires were performing, and more applicant data (IAT and peer competency rating) from an increasing number of users. GROW's Al was poised to deliver ever more robust hiring recommendations to clients. But as Pukuhara's Albecame smarter, what about his human clients? Was it time to educate them on the best ways to use GROW? If so, which of the ways in which his clients were using GROW were likely to be better and less potentially miskin? Fukuhara-san also cautiously saw potential to expand GROW's capabilities. As ICS collected data not just on current recruits but also on the performance of previously recommended his GROW'S AI had the potential to improve not just the hiring process but also the hiring criteria. Was it time to tum on the reinforcement machine learning" Al (Exhibit 4) such that the Al would be able to overrule Some of the client-defined characteristics of an ideal candidate that had not worked well in the past based on how hired individuals ultimately performed at work? In short could Pukulana's Al become smarter than his clients -and how would his clients feel about such a proposition and the ending of control it would require Fukuhara wondered in what ways GROW should grow, dowe Operation (MGT3121 by Gery LGUNY Collegato ary 2011 2018 For the exclusive use of Y. Sherhan, 2018. For the exclusive use of Y. Sherhan, 2018. GROW Ling Artificial Intelligence to Souman Intelligence Exhibit 1 Using Grow 1. Download App OS or Android 2. Complete Basic Protile Information 3 Complete Academic Protile Upload Resume GR w 4. Find friends already on GROW 5 Invite trends not on GROW to join 6. Make or respond to friends feedbacktes uests Source 10 cury documents Marga OT 3121 by Corynow, CUNY Baruch College from January For the exclusive use of Y. Sherhan, 2018. GROW, Ling Artificial Intelligence to fireen Human Intelligence Weighting Users received ratings from multiple evaluators (on average four to five in addition to the self- evaluation, but each evaluation was not weighted equally. Instead. GROW employed a patent- pending probabilistic (Bayesian) machine learning (AI) algorithm that used data of how that competency had been evaluated historically (called priors) along with many data points about an evaluator to determine the posterior likelihood that such an evaluation was genuine and calibrated to reality (Exhibit 4). For example, every query-rubric pair had its own data regarding the amount of time it historically took users to complete that evaluation (from reading the query and rubrics to making the sessment). So, if an evaluator answered much faster than most, that might lead the algorithm to decrease the weight of his evaluation. Similarly, if an evaluator gave more diverse ratings over time- some I'ls and 4's-she might be deemed more discerning and therefore much more reliable than an evaluator who gave all 3's The algorithm also took into account an evaluator's IAT results, as certain personality tendencies (such as conscientiousness) often led to more reliable evaluations. Another key data point was the social graph of the evaluator, an evaluator that gave evaluations for multiple users in various social networks was typically more reliable than an evaluator who only joined the app to give evaluations to one particular sr. Personality Traits The Implicit Association Task was a well-known test in social psychology to reveal attributes and biases people have and the test has been connected to personality traits. In GROW, users progressed through a series of implicit association tasks, in which they drag certain attributes, appearing at the hottom of the screen to the containing the correct attribute(for example, talkative to extroverted) (Exhibit 3). The box containing the correct attribute was either juxtaposed or matched with a concept (mysell, others, and the manner in which the user swiped the moveable attribute was assessed to predict the personality traits of the user. GROW's patented mobile IAT utilized unsupervised machine learning to reveal anomalies in patterns of swiping behavior and better predict personality How Organizations Use GROW After numerous rounds of testing ICs was ready to provide GROW as a software-as-a-service (SAAS) tool for interested firms. There was overwhelming interest. With the advent of online resume submission, digital outreach and apps specifically designed to help students apply effortlessly to many potential employers, applications were skyrocketing the average undergraduate in Japan was sending between 50-100 resumes, or "Entry Sheets" (ES), to companies during job hunting season. But even as companies were flooded with applications, their approach to processing those applications had not changed much they still relied heavily on laborious resume screening and several time-consuming rounds face-to-face interviews. As a result, they simply could not keep up and the quality of their new hires were suffering as result. GROW presented a very attractive solution, although as firms adopted it, it became clear that firms were using the tool in different interesting and even unexpected ways. Septeni Holdings Septeni Holdings (Septeni) was founded in 1990 to develop a human resource consulting business at the height of bubble-era Japan. After the bubble burst Septeni shifted its focus to internet advertising just as the internet was taking off. In 2017, Septeni was one of the largest internet marketing companies in Japan, and it catalyzed growth by adopting unique approaches to recruit and cultivate the best entrepreneurial talent who could then effectively grow subsidiaries and therefore revenues. For the exclusive use of Y. Sherhan, 2018. GHOW. Ling Algemee to the Human A Short History of IGS Prior to starting IS. Fukuhara was a managing director at awt management firm Barclays Global Investors (BGT) where he made investment decisions huwd on computer-driven models (se Exhibit for Pukuhara's work history. The mantra at was "quantify everything and it instilled kuhara with the belief that quantitative judgment would result in greater risk-adjusted retums. As Fukuhara moved up in the organization, he spent time managing the data and more time managing people, and he began to ponder why not qualify personal capabilities as well? Like investors making at management decisions, he noted that people can fail to othvely and perhaps quantitatively) assess themselves and others. Could the advantages of quant Investing applied to evaluating and developing people Following Black Rock's acquisition of HGL and sensing an opportunity to pursue his curiosities for himsel Fukuhana left the firm and founded an educational venture, ICS At first. Pukuhana Tan IGS as a small private cram school in a bustling part of Tokyo. Doing so provided him with a chance to observe firsthand how young people grow, how they respond to feedback and how they learn to their own skills. An avid reader of philosophy educated in both Japan and abroad. Pukuhara designed a novel currkulum of philosophy, critkal thinking, debate, and English as a second language (ESL) which drew interest not just from parents and students in Japan but also from schools and governments to whom Pukuhara consulted. Through the conversations and thinking back to his own experiences with 360 degree feedback at INSEAD and BGI, Fukuhata felt increasingly confident that people's behaviors could change for the better through more frequent foedhack-once or twice a month rather than once or twice a yeart Yet, drawing on his Japanese roots he knew changing honest feedback was uncomfortable for many Japanese, making it an uncommon event. To Fukuhara, the hiring process was a med opportunity to collect utiline, and exchange such fedhack in a systematic way and at a juncture when individuals were likely to act on it Once again Fukuhara saw a connection to his former work. At BGI during his years during his years as an FX and global fined income researcher, he developed early deep learning and other advanced statistical methods to support the quant investment platform. Pukuhara sought to build a similarly scalable technology for feedback in the hiring process. In early 2015, he began holding weekly hrainstormingusions with former colleagues, friends, and researchers to flesh out a tool that would both help students understand their strengths and weaknesses and assist HR in hiring, As a nod toward the desire to help students grow, the tool was named GROW. With marketing support from one of Japan's most prominent media companies. Asahi Shimhun, as well as funds earned by spinning off IGS s education arm, Fukuhara turned to focusing ICS on developing the two-sided GROW platform. GROW Through focus groups with student users and countless meetings with executives at various firms and institutions. IGSs agile team developed GROW as a gamified, mobile consumer app that allowed students to gift cach other feedback on various competencies and discover their personality traits through a modified IAT task. Needing companies to join the platform. IGS added a B2B2C strategy and began providing GROW to HR functions in organizations. Farly clients included mostly tech/manufacturing companies like DNA and Calsonic Kansel, though clients used the service in various ways to identify star players in their firms, to assess those who had already received job offers to join their firms (potentially to place them in certain roles), or to ascertain which subset of the 25 competencies were strategically critical yet consistently underrepresented in employees' test results. In need of more cash to improve the technology, increase client acquisition, and expand the user base (grassroots efforts signed up just a few hundred), Pukuhara set off for meetings with VCs and, by 2 Theodore ay by Yucal Bhetani servo Opension Management IMOT 12) by Cory Labnow. Car-Bartholgeboy 2018 For the exclusive use of Y. Sherhan, 2018. GHOW.Using Artificial telligence to Score Human Intellige 415020 Exhibit3 Implicit Association Test (IAT). GROW's personality assessment The below are based on the Big Five personality traits, developed independently by several groups of researchers in the 20 century, Extroverted You are proactive, dynamic, action-oriented, and always looking for external stimulation through interaction. On the other hand, your actions can be 1 cable Introverted You are reserved and prefer a stable environment. You do not look for external stimulation and find stability within yourself. On the other hand, you can be a cowand can be slow in taking actions. Open You try to obtain new information and expand your knowledge base and 2 experience. On the other hand, you can be unrealistic and can have idealistic ideas Conservative You are content with the current situation and grounded. On the other hand, you do not like changes and tend to stick with the authority and traditions Sensitive You can be sensitive to risk and tend to avoid risk by being prudent. On the other fund, you can get really nervous and emotionally unstable when placed in a 3 Mressful environment You are not influenced by risk and are always emotionally stable. On the other lund, you are insensitive to other peoples' feelings or environment. Cooperative You are a wood listener can understand others, and create cooperative relationships. On the other hand, you tend to think of others too much and tend to be a follower. Therefore, the cooperative person can lack leadership and can tend 4 to be a shrinking violet within the group Independent You can have a strong character, have the ability to generate original and new ides, and lead a group. On the other hand, you have a tendency to be selfish and too competitive You have a strong will and make efforts to achieve a goal every day. On the other 5 hand, you can be a perfectionist and can frustrate yourself and others. Laidback You accept who you are and the environment and are flexible. On the other hand. you can lose your way and can make careless judments In an example of GROW's customised, mobile friendly test concepts of self and others (in orange) are randomly paired with personality traits (in yellow). The user is asked to swipe the moveable yellow box (labeled Tallarive in this example) at the bottom of the screen to the correct attribute (extroverted). Someone who considers themselves extroverted may besitate when associating others with being more talkative. This "hesitation is tracked and analyred in two ways in the pattern of the swipe (straight to the left or in a curved fashion) and in the time tracker (at the bottom of the screen). Established priors can then help determine the posterior likelihood of user's personality type. Source Scompany documents 10 hy by the nerve Operation ManagemMT 3121) het by Cory Larow, CUNY- Baruch Cobagtromy For the exclusive use of Y. Sherhan, 2018. 15.00 GROW. Laing Antintelliger te Screen Human Intelligence To che talent. Septeni had traditionally followed the common Japanese approach of inviting university students to its offices in Tokyo to take part in multiple rounds of group interviews, in which groups of applicants would be monitored as they completed teamwork-based tasks that simulate typical work at Septeni. But that approach had significant drawbacks for Septeni, including the need for students to visit Sepleni in person (which largely excludes students living outside of the Tokyo anual the need for students to be sufficiently aware of and interested in Septent as a potential employer to invest the time and money to join the group interview days (which, as a mid-sized company, was not always the case), and the need for students to perform exceptionally well in the one-shot group interviews, as that was the only chance Septeni had to assess them before making a hiring decision To address the first issue Septeni had created a new online hiring process, composed of one video Interview plus a web based inquiry, for students living outside of the Tokyo area who could therefore opt out of paying expensive travel costs for the interview. That, however, left Septeni without the data it had traditionally collected through the in-person group interviews. Septeni turned to GROW as a substitute for that data. By collecting evaluations from peers throughout a student's time in university, GROW fit well with Septeni's internal evaluation criteria, which was focused on personality traits and on capabilities an individual demonstrated as she or he worked with others over time. In 2016, Septeni provided ICS with data on prior year candidates and interview outcomes to help ICS "train the Al algorithm, from which is developed a supervised machine learning algorithm to accurately predict which candidates--past and future-would pass Septeni's group interviews, Early results with GROW in the 2017 recruiting season were promising. Not only did GROW's predictions dosely match the outcomes of Septeni's internal evaluations, but it doubled the talent Septeni sourced from outside Tokyo, made potentially obsolete the group interview process, and increased Septeni's name recognition among students (as GROW's other clients included well-known firms in Japan). The final outcome a 90% reduction of the overall process effort whilo Sepleni's year-over-year acceptance rate of its job offers jumped four-fold, all with no apparent impact to candidate quality. All Nippon Airways Japan's largest airline, All Nippon Airways (ANA), was consistently rated among the most popular companies in the eyes of students seeking jobs after graduation. To build its pipeline of future senior leaders, ANA sought to identify promising students by screening the tremendous number of applications they received every year. With only a limited number of HR staff, however, ANA feared missing needles in the haystack students with the potential to be future leaders but who were screened out too early in the recruiting process First IGS worked with ANA to prioritive ten competencies it would highly value in its new recruits. Students interested in ANA then used the GROW app have their competencies and personality traits assessed, which was used to create a "total score." Based on all of the other data IGS collected about a student and her evaluators, GROW's Al engine also produced a "confidence score" to rate the degree of confidence IGS had in that total score. ANA then plotted each applicant on a single graph, with "total score" on the x-axis, confidence score" on the y-axis, and the color of the dot representing how far into the screen process the candidate progressed (application received, invited to first round, invited to second round, invited to third round, finalist, received job offer) (Figure 1). In lapar, it was common for companies to hold interview days at their leadquarters and expect all interested candidates to show up in person at their own expense IMT 2121) by Cory Labow. CUNY- Baruch College women For the exclusive use of Y. Sherhan, 2018. GROW, Ling Ancillatelligence to SHuman Intelligence Exhibit 2 Competency Evaluation in GROW Tech.competency was selected from hundreds of competencies and skills reported in the sociales literature on the basis of veral criteria: 1) representation across different countries and culture 21 correlation to success in relevant job functions as verified by cutive search forms and 3) essable by own. The winnowing down of competencies was done by Dr. Mitsuru Kimura of the University of Tokyo An excerpt of each competency, translated from Japanese, follows with Tested BE When a mere complete the duckman web permend en hew the improve 86 . 3 54 comun dement dyty ShanDongTay Cory Labo CNY - Colection 2018 For the exclusive use of Y. GROW:Ling Aftatellite for Human Intellige Figure 1 ANA can use GROW to interview promising candidates it would have seened out Comic Son Total Score SANA .company docum The data realed several insights 1. ANA could use GROW to screen out candidates that were unlikely to make it to a final interview, as candidates with a less than 4 confidence score and less than 5 total score were extremely unlikely to become a "green dot": 2. The chustering of green dots in the upper right of the graph font confidence to the use of GROW as a tool for ANA'S HR staff and 3. There were many students who did not make it past the application screening and yet could not he distinguished from final round interviewees using GROW. To investigate the third insight further. ANA invited around 423 students who had high GROW scores to interview along with the candidates who passed their traditional process. To ANA's surprise, some of the GROW students invited received perfect scores in the interviews, leading HR to realize that by using GROW data in a supervised machine learning manner, GROW could help surface promising candidates ANA would have missed otherwise As a result, rather than focusing on screening students out ANA decided to use GROW to screen students in, further developing and fine-tuning its inclusion criteria. With these parameters, ANA could use GROW to more accurately target clusters of students with high potential to advance through their recruitinent process Mitsubishi Corporation One of Japan's largest and most storied enterprises, Mitsubishi Corporation (MC) operated businesses in diverse industries, including industrial finance, metals, machinery, chemicals, new energy, infrastructure, finance, technology, daily living essentials, and more. In 2017, MC's activities extended far beyond its traditional trading model to include active participation in the management of its businesses. While the firm historically had few issues attracting Japan's best and brightest graduates (it was often number one in popularity among new graduates), the evolution of its business model had This documents and only by the Brio Operations Management (MGT31 taught by Cory Lahanow, CUNY.Baruch Chege tromy 2018 For the exclusive use of Y. Sherhan, 2018. HARVARD BUSINESS SCHOOL 9-418-020 STRAN FERNSTEIN PAUL MCKINNON PAUL YARAR GROW: Using Artificial Intelligence to Screen Human Intelligence Masahiro Pukuhura, founder and CEO of Tokyo-tused people analytics startup Institution for a Global Society (CS), took a break from poring over data to question how it would be the used. In the seven years since Fukuhara had founded is, its solution to evaluate job candidates - "GROW-had grown quickly, and the wide variety of ways clients used it simultaneously excited and concerned him GROW. an iOS and Android app (Exhibit 1), consisted of two proprietary components a competency assessment and a personality assessment. To assess competencies GROW employed a peer-feedback tool to reveal 25 specific competencies which ICS had chosen based on extensive social science research (Exhibit 2). To assess personality, GROW employed a gamified version of the Implicit Association Test (IAT), an established assessment of hidden bias in social psychology, that individuals could play on mobile devices (Exhibit 3). While neither component was particularly novel, what GROW did with them was it used artificial intelligence (AI) learning algorithms to analyze every speck of assessment data from both the candidates and the evaluators, looking for patterns to improve its ability to accurately screen candidates over time (Exhibit 4). In place of human "intuition," GROW used "big-data"-disparate data points across many people to develop a scientific, objective, and constantly improving engine to recruit screen and develop human capital. So far, it seemed to work. For example, in a test of GROW, one client had both its HR professionals and GROW evaluate the same 200 students. GROW not only surfaced nearly the same top 50 candidates (the two lists of 50 were statistically indifferentiable), but more importantly it did so with specific data-based competency-based justifications. As of June 2017, GROW had 74,000 users, including students at both prestigious and lesser-known universities. Clients included Mitsubishi Corporation, All-Nippon Airways (ANA), Septeni, DeNA, Rakuten, AXA, and many others. Even government entities like Japan's Ministry of Economy, Trade ke Industry (METI) and the UAE were getting involved. The widespread interest in GROW was both an opportunity and a challenge. On the one hand, it provided Pukuhata with a growing base of users, data, and institutional support. On the other hand, Fukuhara wondered if he should play a stronger role in strategically focusing the use of GROW where it was likely to have the most meaningful (and least potentially misleading) impact. Plemeniele Pail Menon PYMA perpow whence of Nobuo 12 Ni Tarand Almosthenewed and approprio by company de Funding for the deleted provided by Harvard School and by the company are developed Cormichon Follows of Harvard College Tonder weer que presion to produer material call 1-800-2005 wird scholing MA pelowww.libeparvard.edu. This publication may the dig.photocopied www produce posuded, without the permission of Harvard School ta multorte brune cry by Semannsenice Operations Management (MGT 3121) taught by Cory Labanow. CUNY - Baruch College from January 2012 2018 For the exclusive use of Y. Sherhan, 2018. 411.00 GROW. Using Artificial Intelligence to See Human Intelligence Exhibit 4 How the Artificial Intelligence in GROW Works 4+ Emites Par En User Dette Captured buth GROW App Competency Evaluator Ratings of User Personality IAT User Personality IAT Umupervised Machine Learning Al pof the Supervised Machine Learning Al ww Multiple Per Rating of User Candidate Recommendation Bry 3 L! - Duta Soome Reinforcement Machine Learning AI Be -GROW Date dom - Company Data Current) Artificial Intelligence (AD) a) Aitical Intelligence AD Come Recommendati Hiring Company) cally Sour Crewritt 11 Operations Management MOT 5121) taught by Cory Labanow. CUNY Baruch College tomayo