Understanding Conversational Programmers A Perspective From The .
Understanding Conversational Programmers:A Perspective from the Software IndustryParmit K. Chilana1, Rishabh Singh2, Philip J. Guo323Management SciencesRiSE groupComputer ScienceUniversity of WaterlooMicrosoft ResearchUniversity of RochesterWaterloo, ON CanadaRedmond, WA USARochester, NY ochester.edu1ABSTRACTRecent research suggests that some students learn toprogram with the goal of becoming conversationalprogrammers: they want to develop programming literacyskills not to write code in the future but mainly to developconversational skills and communicate better withdevelopers and to improve their marketability. To investigatethe existence of such a population of conversationalprogrammers in practice, we surveyed professionals at alarge multinational technology company who were not insoftware development roles. Based on 3151 surveyresponses from professionals who never or rarely wrotecode, we found that a significant number of them (42.6%)had invested in learning programming on the job. Whilemany of these respondents wanted to perform traditionalend-user programming tasks (e.g., data analysis), wediscovered that two top motivations for learningprogramming were to improve the efficacy of technicalconversations and to acquire marketable skillsets. The maincontribution of this work is in empirically establishing theexistence and characteristics of conversational programmersin a large software development context.Author KeywordsConversational programmers; programming literacy; nonCS majors; technical conversations.ACM Classification KeywordsK.3.2 Computers and Education: Computer and InformationScience Education—literacy, computer science education.INTRODUCTIONMomentum around the importance of programmingliteracy [31] has catalyzed several specialized initiatives toteach coding skills to a broad audience. From introducingPermission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. Copyrights forcomponents of this work owned by others than ACM must be honored.Abstracting with credit is permitted. To copy otherwise, or republish, topost on servers or to redistribute to lists, requires prior specific permissionand/or a fee. Request permissions from Permissions@acm.org.CHI'16, May 07-12, 2016, San Jose, CA, USA 2016 ACM. ISBN 978-1-4503-3362-7/16/05. 15.00DOI: ng to elementary school children [5] tolaunching introductory programming university courses forstudents outside computer science (CS) [11] to creatingfree massive online programming courses (e.g., Coursera,edX, Udacity) hundreds of academic, government, andindustry efforts around the world are trying to prepare atech-savvy workforce of the future.Learning and teaching programming has also been a keytheme in human-computer interaction (HCI) andcomputing education research for over three decades [25].For example, numerous projects have contributed insightsinto the struggles of novice programmers [13,16], proposeddesigns for improving the usability of complexdevelopment environments [26,28], and even invented newprogramming languages that simplify programmingconcepts for learners of all ages [29]. While these researchfindings and experience reports from classrooms havehelped us better understand the barriers to learningprogramming, many of the insights are based on theassumption that learners will eventually write code (e.g., asa professional developer or a domain-specific end-userprogrammer [20]). But, is this always the case?Recent research shows that some students in non-CS fieldssuch as management may not aspire to write code as anend-user programmer or a professional programmer, butare still strongly interested in taking programming classes[6]. These students were termed conversationalprogrammers because they wanted to develop onlyconversational skills in programming literacy to be able to:1) aid technical conversations with professional softwaredevelopers in the future; or 2) enhance their marketabilityin the software industry.Unfortunately, beyond this classroom study and informaldiscussions by practitioners [10,22], we know little aboutconversational programmers in today’s software industry.To what extent do these conversational programmersactually exist in practice? How do conversationalprogrammers perceive programming literacy? How do theyinteract with software developers? What motivates them toacquire programming skills, if at all?In this paper, we investigate the prevalence and perceptionsof conversational programmers at XYZ, a largemultinational software company. We conducted a large-
scale survey targeting professionals in roles outside ofsoftware development (e.g., sales and marketing, customerrelations, managers, lawyers, user experience researchers,designers, and others) and encouraged participation fromrespondents who had no formal training in CS. Based onanswers from 3151 respondents who never or rarely wroteany code, we found that close to half (42.6%) of them hadinvested in learning programing skills on the job. Whilemany of these respondents were learning programming tosupport small end-user programming tasks, such asanalyzing data (24.5%) or creating prototypes (20.6%),over half of our respondents’ motivations fit the definitionof conversational programmers [6]—they were learningprogramming to improve their technical conversations withdevelopers and customers (29.6%) and viewedprogramming literacy as a marketable skill for their overallcareer (22.7%). Furthermore, we found that many of theconversational programmers who engaged in end-userprogramming tasks were, in fact, using their artifacts toimprove communication with developers.The main contribution of this work is in providingempirical insights that establish the existence andcharacteristics of conversational programmers in thecontext of a large software development company. Ourfindings complement existing insights about professionaldevelopers [21,30] and end-user programmers [3,20,27,32]and raise important questions about how to teachprogramming so that students not only learn the mechanicsof writing code, but also learn techniques for establishingcommon ground [8] in conversations with developers. Wehope that our analyses will be useful for HCI and computingeducation researchers and practitioners to consider as theychart the future of training a programming literate society.RELATED WORKTo contextualize our insights about conversationalprogrammers, we draw upon a variety of research fromHCI, computing education, and software engineering.End-user programmers vs. conversational programmersFor several years, there has been a push to teachprogramming skills to students who are not necessarilygoing to be professional software developers. For example,numerous efforts have been created to teach programmingto students in Biology, Fine Arts, and Business [11,12,36]so that they can not only enhance computational thinkingskills [37], but also have skills to design or customizesoftware to solve specific problems [3,27]. Commonlyknown as end-user programming, this sub-disciplineconcerns "a set of methods, techniques and tools that allowusers of software systems, who are acting as nonprofessional software developers, at some point to create,modify, or extend a software artifact" [20]. In fact, researchshows that the population of end-user programmers isgrowing and even outnumbers professional programmersmany fold [32].On the other hand, a recent study [6] suggests that noteveryone who learns to code is necessarily going to be anend-user programmer or a professional developer. Somelearners have the goal of becoming conversationalprogrammers: they want to develop programming literacyskills not to write code in the future but mainly to developconversational skills and communicate better withdevelopers in the future and to improve their marketabilityin the software industry [6]. Our main goal in this paper wasto assess the prevalence of conversational programmers inan industrial setting: are there people who learnprogramming on the job even when they are not required towrite code? What motivates them? While our findingscorroborate prior work and show that end-userprogramming tasks (e.g., analyzing data, prototyping) arestill a major reason why professionals learn programmingon the job, we also found that conversational programmersare, in fact, prevalent in industry. And, even the end-userprogramming tasks that our respondents described wereoften carried out in the context of improvingcommunication about design decisions.Understanding and lowering barriers to programmingGiven the momentum around “programming for all”[10,29,31], numerous studies from HCI and computingeducation have contributed insights into the struggles oflearning programming in class and on the job. Some havefocused on lowering the barriers to using programmingtools and development environments [16,19] and focus onthings such as syntax issues and program flow. Others haveeven proposed new programming languages, such asScratch [29], that simplify programming concepts forlearners of all ages. There is also a long history ofdesigning graphical or visual programming environments(e.g., programming by example [24], and visualprogramming [33]).The central goal of these works has been to understand thechallenges people have in writing code and in developingtools or approaches to address these challenges. Theassumption that the end goal is to write code has influencedmost pedagogical strategies and tools that aim to lower thebarriers to programming. However, our findings aboutconversational programmers in industry show that eventhough they never or rarely wrote code on the job, they stilllearned programming to improve their technicalconversations. We raise several questions around what itmeans for us to teach programming to this class of studentsand to design new tools and programming environmentsthat facilitate the conversational aspect of learningprogramming.Social aspects of developing softwareAnother class of research that is relevant to our study isrelated to the social and collaborative aspects of softwaredevelopment. Over the years, there have been a number ofstudies on understanding development practices [1,15] andinformation needs of software developers [18]. Since
software decisions about design and implementationrequire discussions and coordination, getting everyone tohave a shared understanding or establishing “commonground” [8] is important. Common ground is theknowledge and awareness that participants have incommon when they are communicating with each other[8]. Previous studies suggest that to establish this commonground in software teams, developers need to be trained notonly in programming, but also in communication skills and“people skills” [1,10,15,21].But, these studies are mainly based on perspectives ofsoftware developers, which represent only one side of thepartnership in software teams. Our results onconversational programmers (who came from roles inmarketing, sales, management, UX/Design, customerrelations, and others) suggest that while communicationskills are important, they are not perceived to be enough—our study adds insights about how programmingknowledge is used as a conversational medium to establishcommon ground. We also illustrate other strategies thatconversational programmers had developed to participatein technical conversations, such as involving experts,engaging in the conversations, and finding informationonline.In summary, while we have many insights into the work ofend-user programmers, barriers associated with learningprogramming, and the social and collaborative aspects ofdeveloping software, to our knowledge, no study haspreviously considered the perspectives of conversationalprogrammers in industry.METHODTo investigate our research questions about conversationalprogrammers, we decided to take a survey-based approach,as it would allow us to capture perspectives on a large scaleacross demographics, job roles, and college majors. Ourresearch site was XYZ, a multinational software companywith over 80,000 employees. XYZ specializes in over twodozen personal, enterprise, mobile, gaming, and cloudcomputing applications.In mid-2015, we sent an online survey to a sample of XYZemployees who were not currently working as softwaredevelopers but who worked in roles where they were likelyto discuss technical concepts with either developers orcustomers (i.e., they were potentially conversationalprogrammers). As per the definition of conversationalprogrammers, the survey instructions encouragedparticipation from people who did not major in computerscience, computer engineering, software engineering,information technology and related fields. We received3682 responses ( 14% response rate1). One possible factorthat lowered the response rate was that many of the1For confidentiality reasons, we cannot reveal exact job rolenames, proportions, or numbers of personnel at XYZ.sampled employees still had some formal computer sciencetraining since XYZ was a software company, so they didnot take our survey.We developed our survey questions via an iterativeapproach, starting with a pilot survey sent to 1000randomly-selected XYZ employees. Our response rate forthe pilot survey was 24.5%, which was higher than ouractual survey since the pilot did not specifically targetpeople without CS training and many respondents ended upbeing CS majors. We used feedback from the pilot to refinethe wording of questions and to finalize the choice labelsfor the multiple-choice questions.The survey instrumentOur survey consisted of 13 multiple-choice and 3 openended questions. The first 4 questions were aboutdemographics: job role, years of experience in that role,college major, and gender. The next 4 questions askedabout programming background and education:Are you comfortable writing code in one or moreprogramming languages? If so, which ones?Which programming language(s), if any, do you need toknow to do your job?How many courses involving computer programming didyou take as part of your formal education (undergraduateand graduate)? Choices: 0, 1, 2-4, 5 or more.Choose all of the ways (if any) in which you have learnedprogramming to help with your job. Choices: None, books,websites, colleagues, online courses, company-offeredtraining, part-time university courses, other.In a subsequent open-ended question, we further askedthem to explain what motivated them to learn programmingon the job.The next 3 questions asked how often people wrote orcommunicated about code. The five choices were: {daily, afew times per week, a few times per month, rarely, not at all}How often do you write code as part of your current job?How often do you have technical conversations (e.g., aboutcode, software architecture, programming concepts) withsoftware developers as part of your job?How often do you have technical conversations (e.g., aboutcode, software architecture, programming concepts) withcustomers (either internal or external) as part of your job?In another open-ended question, we used the critical incidenttechnique [4,9] and asked respondents to describe a recentmemory of a technical conversation with a developer or acustomer that presented a challenging situation and how theytried to resolve that situation.The next 2 questions were about the perceived marketabilityof programming skills. Respondents answered each on a 7point Likert scale from Strongly Disagree to Strongly Agree:In a recent study, first-year non-computer-science studentsindicated that learning programming made them more“marketable” in today's job market, even if they were notgoing to write code for their job [6]. Based on your own
Finally, we had an open-ended question asking respondentsto give advice to future employees in similar roles whowould be working closely with software developers.In designing our survey, we realized that some people mayalso learn programming on their own or throughextracurricular activities. So, we asked respondents tospecify what programming languages, if any, they werecomfortable writing code in. We found that just over athird of the respondents (34.0%) were comfortable writingcode. The most common programming languages citedwere HTML, SQL, Visual Basic, CSS, and C#.Data overview and analysisProgramming on the jobexperiences, to what extent do you agree or disagree withthese students' opinions?To what extent do you agree or disagree with the followingstatement: If I were to start my education all over again, Iwould have taken more programming courses.Our initial 3682 respondents came from a variety of jobroles, including those in sales, marketing, management,business operations, legal affairs, design, and customerrelations1. Respondents had diverse levels of experience intheir roles: 29.2% had 5 or fewer years of experience, 41.5%had 6 to 15 years, and 28.8% had more than 15 years ofexperience1. 62.9% identified as male, and 35.3% as female.They came from over 40 different undergraduate management, marketing, psychology, design, and HCI (only0.08% majored in computer science/engineering/informationtechnology). When we asked respondents about how oftenthey wrote code as part of their current job, anoverwhelming majority (86.6%) responded with either “notat all” (71.5%) or “rarely” (15.1%). We filtered our datasetand the rest of the analysis to include only the 3151respondents who never or rarely wrote code on the job (wealso eliminated 38 respondents (1.0%) that did not answerthis question). Therefore, our findings represent perspectivesof non-CS majors who were not in engineering roles andwere not required to write code on the job.Of those who filled out the survey, there was a highresponse rate for the three open-ended questions ( 50%).We used an inductive analysis [34] approach to devise aclassification scheme for each of the three questions. Allthree of the authors were involved in iteratively developinga coding scheme. To formally assess the reliability of ourcoding scheme and to measure agreement between allcoders, we computed the Fleiss Kappa on a uniformrandom sample of 150 responses from each of the threeopen-ended questions. We iterated on the classificationuntil we found moderate to strong agreement by the threecoders (! 0.73).EXPERIENCE WITH PROGRAMMINGTo understand the context of our respondents’ perceptions,we look at their experience with programming throughformal education or on-the-job training.Programming courses and prior experienceFirst, we were surprised to learn that over half of ourrespondents (54.2%) had taken at least one programmingcourse as part of their formal training, even though themajority were not CS or engineering majors. Since manynon-CS programs are increasingly requiring or stronglyencouraging students to learn introductory programming[11,12,36], perhaps this statistic is just demonstrating thisgrowing educational trend.When asked about programming languages used on thejob, the majority (84.2%) said that they did not use anyprogramming languages on the job. The other 15.8% of therespondents mentioned using a few different languages,including SQL, C#, HTML, shell scripting, and JavaScript.We also asked respondents to select all of the ways inwhich they had learned a programming language to helpthem with their job (if any). Interestingly, even thoughmost of the respondents never or rarely wrote code, closeto half of them (42.6%) had still invested in learningprogramming on the job. The learning efforts ranged fromconsulting programming-related websites, to readingbooks, to learning from colleagues, to enrolling inspecialized courses and training (Table 1).Motivations for learning programming on the jobTo probe into why people would invest time and effort inlearning programming when they did not need to writecode as part of their job, we asked them an open-endedquestion about what motivated them to learn programmingon the job. We received 1759 written responses for thisquestion, constituting a response rate of 55.8%. Throughour analysis of the open-ended responses, we came up withthe classification scheme shown in Table 2 and categorizedeach of the responses.As shown in Table 2, the most common motivation fortrying to learn programming on the job was to improvetechnical conversations with developers and customers(29.6%). Other motivations for learning programmingdescribed by respondents can be categorized as traditionalend-user programming tasks (e.g., analyzing data, creatingprototypes, improving efficiency) that have been discussedextensively in prior work [20,32]. But, in many cases, ourrespondents were learning to write code for these end-userprogramming tasks to also improve some aspect of theircommunication with developers. An additional 22.7% ofthe respondents learned programming for their personalStrategyPercentWebsitesBooksFrom colleagues32%28%24%Online coursesTraining offered by companyPart-time college/university courses18%14%12%Table 1. Top resources for learning programmingon the job (*note that the overall total is 100% since someresponses fell into multiple categories)
Motivation CategoryPercentimprove conversations withsoftware developers or customers29.6%analyze data24.5%personal growth and marketability22.7%Example ResponseI wanted to make sure I could hold my own in discussions with softwaredevelopers, and I wanted to be able to talk about our products and platformwith external customers.I have learned some scripting languages which has helped me analyzedata. Nothing too difficult and the motivation was to get the information tobetter understand my customer base.Kids come out of school these days with such skills I can only compete with ifI keep my knowledge fresh and diverse, my skills polished. wanted to illustrate the ideas I had and sometimes showing it is a betterway to illustrate things. Easier to communicate with other developers.improve efficiency (e.g., automateWanted to build some simple tools to speed up my job and to understand13.0%repetitive tasks)better how the developer was coding a featureTable 2. Classification of respondents’ self-reported motivations for learning programming on-the-job (*note that the overall totalis 100% since some responses fell into multiple categories)build prototypes or demos20.3%growth and to stay relevant and marketable in the softwareindustry. We discuss each of these motivations below.code reviews to understand or debug particular issuesduring conversations:TECHNICAL CONVERSATIONSDEVELOPERS AND CUSTOMERSI tend to have most of my technical conversations around bugbashes - the test scenarios, what would constitute a bug, how toprioritize the bugs and what should be considered a blocking bugor not.WITHSOFTWAREThe top motivation for learning programming on the job(29.6%) was to be able to communicate better withsoftware developers and customers. Our analysis furtherrevealed that almost half of the respondents (47.6%)regularly participated in technical conversations withdevelopers (few times a day, week, or month) and anotherlarge number of respondents (41.4%) regularly took part intechnical conversations with customers.Below we discuss some of the reasons why respondentslearned programming to help them with their technicalconversations. We also describe the context of theseconversations and the other strategies that the respondentshad developed over the years to tackle these technicalconversations.Programming literacy for improving conversationsMost of the respondents who were making an effort tolearn programming to improve technical conversationswere not interested in writing code, but rather in learningthe common phrases and terminologies to follow along inthe conversation at a higher level:I don't necessarily need an in depth technical knowledge and noprogramming knowledge is required however it would be usefulto have a basic understanding at least to be able to contribute tohigh level conversation.I didn't really need it [coding] but felt it would help me with the'bigger picture.’In contrast, some respondents wanted to understandconversations at a lower level and understand references todifferent parts of the software architecture being discussed: to really know an area well and be able to communicate withthe developers it's great to know what the process of making codechanges takes and how the engine is architected. Granted it'slarge and you'll never be an expert in every area but at leastwhen having discussions and someone mentions terms you knowwhich segments of the code they're referring to.Some respondents described how they would participate inOther respondents who did not have training in writingcode still found it valuable to understand the basic conceptsso that they could better advocate for design orimplementation changes in their conversations withdevelopers:I don't know how to write code, but I often, several times a week,need to look at code and discuss changes with developers. Itreally helps that over the years I have figured out basic HTML sothat I can describe what needs to happen with devs and we canunderstand each other.Some respondents wanted to understand programming sothat they could have more empathy in conversations withexternal technical customers, including developers,managers, technical support, data specialists, installationsupport, and others:I decided to learn how to do some basic programming so that Icould have a better sense of what my customers go through inorder to implement a system.In summary, the conversational programmers in our studyfound it helpful to learn programming so they could betterunderstand technical vocabularies, customer needs, anddevelopers’ decisions and constraints during technicalconversations. Since these conversations are important forsoftware design and implementation decisions, we probedinto better understanding the context of these conversationsand other strategies used by respondents that did notinvolve learning programming on the job.Other strategies for improving conversationsFor our open-ended question about technical conversations,we received 1848 responses, constituting a response rate of58.6%. We analyzed each of these open-ended responsesand eliminated about 8.3% of the responses that did notspecifically answer the question. Another 7.8% of therespondents explicitly stated that technical conversations
Strategies for Technical ConversationsPercentrelying on developers or other experts to do thetranslation (internal or external to the team)39.3%engaging in the conversation (e.g., asking lotsof questions, going to the white board, focusingon the business/ customer priorities)finding information on the topic (e.g., bysearching online, books, internal documents)being upfront about the level of technicalknowledge26.0%8.9%4.9%Table 3. Respondents’ strategies for technical conversationswith developers and customers.with customers or software developers did not present anynotable challenges because of their experience or pasttraining in programming:Engaging in the conversation: Another common strategy(26.0%) had to do with making efforts to engage in thetechnical conversations. For example, many respondentsexplained that the best strategy for them was to ask thedevelopers to slow down or simplify the concept they weretrying to get across:During an architecture review.an engineer started talking tootechnical. I didn't understand what he was talking about, so Isimply asked him to describe in plain language - which hedid I've generally found that asking engineers to talk in plainlanguage solves the problem.Many other respondents tried to steer the conversation tomake it more customer or business-focused rather thanbeing bogged down in code:I have been working with XYZ for the last 15 years. That gave mean ample time to understand the architecture and API which iswhat mostly [gets] discussed In my position, bringing the conversation up a level or talking ata high level is more important than the details. At least at first.In this situation, I asked to regroup and make sure we all alignedon the high level points that I could understand.The remaining responses described various technicalconversations, any challenges that they presented, and howthey were resolved. Below, we focus on the remainingresponses that explicitly described a strategy (other thanprogramming) that respondents used when having technicalconversations. Table 3 shows our classification of the topconversational strategies and we discuss them below.While many respondents were able to participate better byengaging in the conversation, some did talk about tradeoffsand feeling like they were derailing the conversation orslowing it down by asking too many questions. Also, somerespondents noted that it was helpful when developerscould also understand the larger business and customercontexts and steer the technical details accordingly.Relying on technical experts: A common strategy(39.3%) described by respondents was seeking expertise ofsoftware developers or other technical experts to translateunfamiliar terminologies. For example, some respondentsdescribed cases where they would seek help from expertsfollowing a conversation:Finding information on the topic: Some respondents(8.9%) also described how they would invest time insearching online or reading up internal documents either toprepare for a technical conversation or to follow up after aconversation:I was in a meeting where there was some heavy techno jargonbeing thrown around, so after the meeting I had a quick call withone of the attendees to put it into English, because part of my jobis also explaining these conce
programming were to improve the efficacy of technical conversations and to acquire marketable skillsets. The main contribution of this work is in empirically establishing the existence and characteristics of conversational programmers in a large software development context. Author Keywords Conversational programmers; programming literacy; non-
Non-programmers will face more challenges with manual service composition compared to programmers. Hypothesis 1e (H1e). Non-programmers will hold a more negative perception about manual composition compared to programmers. RQ2: What are the attitudes of non-programmers when a software tool is "taking over" their design by
be effective in traditional keyboard or touch-based exploratory search scenarios, by and large they are inappropriate to support exploratory search in a conversational setting on mobile devices. Instead, we argue that an approach based on interactive storytelling is called for to support conversational exploratory search. 3 CONVERSATIONAL .
The Conversational Italian for Travelers Audio Guide is a separate textbook in two volumes (which can also be downloaded in MP3 format from this web site), with material that corresponds to and expands each chapter in the Conversational Italian for Travelers textbook. Listen as native
ment levels with Zhorai's conversational interface. Although prior knowledge of conversational agents and other factors may impact a child's ability to understand, we hypothesize that all children can benefit significantly from conversational interaction with Zhorai and the visualizations. From that, we form two specific hypotheses: 1.
1. Teaching with a Multiple-Perspective Approach 8 . 2. Description of Perspectives and Classroom Applications 9 . 2.1 Scientific Perspective 9 . 2.2 Historical Perspective 10 . 2.3 Geographic Perspective 11 . 2.4 Human Rights Perspective 12 . 2.5 Gender Equality Perspective 13 . 2.6 Values Perspective 15 . 2.7 Cultural Diversity Perspective 16
The evaluations involved ten programmers and ten non-programmers in Brazil, performing tasks on a system developed with Google's Blockly [20] and a simulator to support debugging. The results include a thematic analysis of 247 problem instances (80 from programmers and 167 from non-programmers). The remainder of this paper is organised as .
Q&A websites, programmers describe the technical challenges they face using a wide range of questions to seek help and advice. These posted questions give researchers the opportunity to highlight the knowledge needs of programmers [2]. Researchers [2] have also used the posted SO questions to identify how programmers interest in a topic evolves .
Cambridge English Qualifi cations Young Learners tests The Cambridge English Qualifi cations Young Learners tests are for learners of English between the ages of 7 and 12. The tests are comprised of three levels: Pre A1 Starters, A1 Movers and A2 Flyers. These tests are designed to take learners from beginner level up to CEFR level A2.
