Scarlett R. Miller Accepted Manuscript Not Copyedited
Journal of Mechanical Design. Received June 30, 2018;Accepted manuscript posted November 01, 2018. doi:10.1115/1.4041857Copyright (c) 2018 byASMEJournalof Mechanical Design Special Issue: Selected Papers from IDETC 2018.Exploring Biases Between Human andMachine Generated DesignseditedChristian E. LopezIndustrial and Manufacturing EngineeringThe Pennsylvania State University,State College, PA 16802e-mail: cql5441@psu.eduMem. ASMECoottNscripManuConrad S. Tucker1Engineering Design andIndustrial EngineeringThe Pennsylvania State University,State College, PA 16802e-mail: ctucker4@psu.eduMem. ASMEpyScarlett R. MillerEngineering Design andIndustrial EngineeringThe Pennsylvania State University,State College, PA 16802e-mail: shm13@psu.eduMem. ASMEedABSTRACTThe objective of this work is to explore the possible biases that individuals may have towards theptperceived functionality of machine generated designs, compared to human created designs. Towards thisceend, 1,187 participants were recruited via Amazon Mechanical Turk to analyze the perceived functionalAccharacteristics of both human created 2D sketches as well as sketches generated by a deep learninggenerative model. In addition, a computer simulation was used to test the capability of the sketched ideasto perform their intended function and explore the validity of participants’ responses. The results reveal1Corresponding author. 213 N Hammond Building, University Park, PA 16802, USAChristian E. Lopez, Scarlet S. Miller, and Conrad S. TuckerMD-18-1525Downloaded From: .org on 11/06/2018 Terms of Use: http://www.asme.org/about-asme/terms-of-use1
Journal of Mechanical Design. Received June 30, 2018;Accepted manuscript posted November 01, 2018. doi:10.1115/1.4041857Copyright (c) 2018 byASMEJournalof Mechanical Design Special Issue: Selected Papers from IDETC 2018.that both participants and computer simulation evaluations were in agreement, indicating that sketchesgenerated via the deep generative design model were more likely to perform their intended function,compared to human created sketches used to train the model. The results also reveal that participantswere subject to biases while evaluating the sketches, and their age and domain knowledge were positivelyeditedcorrelated with their perceived functionality of sketches. The results provide evidence that supports thecapabilities of deep learning generative design tools to generate functional ideas and their potential topyassist designers in creative tasks such as ideation.Co1. INTRODUCTIONotRecent advancements in generative design, topology optimization, and deeptNlearning algorithms, are enabling designers to integrate computational tools into thescripdesign process at an increased pace [1]. Researchers argue that as these computationaltools become more efficient at creating novel and functional ideas, they will fosterdesigners’ creativity. Hence, both machines and designers will co-create solutions thatManusurpass each of their independently created ideas [2].Deep learning algorithms are being implemented to automatically generate neweddesign ideas [3,4]. Though an idea needs to be new and novel to be considered creative,ptit also has to meet its intended functionality and be useful [5]. During the latter stagesceof the design process, designers create CAD models and implement advancedAccomputational methods to test the functionality of their design ideas. However, duringthe early stages of the design process, rough 2D sketches are typically the primarycommunication source of ideas [6]. During these stages, designers use their experienceand domain knowledge to ensure that their new ideas are relevant to the designChristian E. Lopez, Scarlet S. Miller, and Conrad S. TuckerMD-18-1525Downloaded From: .org on 11/06/2018 Terms of Use: http://www.asme.org/about-asme/terms-of-use2
Journal of Mechanical Design. Received June 30, 2018;Accepted manuscript posted November 01, 2018. doi:10.1115/1.4041857Copyright (c) 2018 byASMEJournalof Mechanical Design Special Issue: Selected Papers from IDETC 2018.problem at hand. Similarly, in the literature, crowdsourcing methods have beenimplemented to assess the ability of generative computational tools to produce newdesign ideas [3,4]. Recently, researchers have started exploring the functionality of 2Deditedsketched ideas generated by computational tools using human raters [7]. Ifcomputational tools are to co-create new products and solutions alongside designers,their ability to produce not only novel, but also functional ideas, needs to be furtherpyexplored. In this work, the term “computer generated” is used as an encompassing termCoto represent various deep learning-related methods of automated design generation.otThe ability to generate creative ideas is an insufficient condition for innovationtNbecause decision-makers need to not only generate, but also select creative ideas forscripinnovation to occur [8]. However, studies have shown that gender effects can influencethe idea selection process [9,10]. Similarly, the educational level, experience, andManudomain knowledge of individuals have been related to individuals’ risk attitudes anddecision-making processes [11,12], while age has been related to technology adoption,acceptance, and perceived usability [13–15]. Hence, as designers integrateedcomputational tools to assist in the design process, their possible bias towardsptcomputer generated and human created ideas, as well as the potential confoundingceeffects of their demographic characteristics and domain knowledge, need to beAcexplored. In light of this, the authors of this work present a crowdsourcing method toexplore the perceived functional characteristics of 2D design sketches created byhumans and 2D design sketches generated by a deep learning generative model, as wellas the effects of participants’ demographic characteristics and domain knowledge onChristian E. Lopez, Scarlet S. Miller, and Conrad S. TuckerMD-18-1525Downloaded From: .org on 11/06/2018 Terms of Use: http://www.asme.org/about-asme/terms-of-use3
Journal of Mechanical Design. Received June 30, 2018;Accepted manuscript posted November 01, 2018. doi:10.1115/1.4041857Copyright (c) 2018 byASMEJournalof Mechanical Design Special Issue: Selected Papers from IDETC 2018.their perceived functionality of design sketches. Moreover, computer simulation is usedto test the capability of the sketches to perform their intended function and test theeditedvalidity of participants’ responses.2. LITERATURE REVIEW2.1 Generative designpyGenerative design methods have captured the interest of both the design researchCoand industry communities [16,17]. In Chandrasegaran et al. [1], the authors present aotreview of some of the challenges and future direction for computational support toolstNused in the product design process. Recently, designers have started to integrate deepscriplearning models into their generative design methods. Deep learning models are a classof hierarchical statistical models composed of multiple interconnected layers ofManunonlinear functions [18]. Designers have gained a particular interest in Recurrent NeuralNetworks (RNNs) [19,20] and Generative Adversarial Networks (GANs) [3,4,21]. RNNsare deep learning models that contain multiple interconnected hidden layers. Theedhidden layers in an RNN are able to use information from their previous state via aptrecurrent weight layer, which allows them to have a recollection of their previous statesce[22]. GANs are deep learning generative models composed of a generator and aAcdiscriminator network. For example, the generator can be trained to generate newimages that the discriminator classifies as “real” images (i.e., drawn from the samedistribution as the training dataset). In contrast, the discriminator is trained to detectthe generator’s output images as being “fake” (i.e., classify images produced by theChristian E. Lopez, Scarlet S. Miller, and Conrad S. TuckerMD-18-1525Downloaded From: .org on 11/06/2018 Terms of Use: http://www.asme.org/about-asme/terms-of-use4
Journal of Mechanical Design. Received June 30, 2018;Accepted manuscript posted November 01, 2018. doi:10.1115/1.4041857Copyright (c) 2018 byASMEJournalof Mechanical Design Special Issue: Selected Papers from IDETC 2018.generator as being drawn from a distribution other than the training data) [23]. Thisiterative game between the generator and discriminator results in GANs being capableof generating designs that are different from the training dataset (i.e., unique at a pixeleditedlevel), while still maintaining some degree of similarity (see [22,23] for additionaldetails).Deep generative methods have been used to help in the representation of thepydesign space. For example, Burnap et al. [3], train a deep generative model with aCodataset of automotive designs able to generate new design ideas that morphedotdifferent body types and brands of vehicles. Dosovitskiy et al. [24] train a deeptNgenerative model to generate new 2D images of chairs. Kazi et al. [6] implement deepscripgenerative models into their DreamSketch tool. The DreamSketch tool takes as input, arough 2D sketch, and generates multiple augmented solutions in 3D. Recently, Chen etManual. [20] present a modification of Ha and Eck’s Sketch-RNN model [19] capable ofrecognizing and generating 2D sketches from multiple classes. As highlighted by theauthors, this model has the potential to help with creative tasks [20]. Deep generativeedmethods have also been implemented to increase the veracity of big-data pipelines byptgenerating new images [4]. However, an inherent challenge of these generativecemethods is that their objective to create new design ideas that still maintain a degree ofAcsimilarity with the training data used are conflicting and challenging to evaluate. Whilestudies have implemented pixel-level Euclidean distance and structured similarityindices to evaluate these methods, in many cases, these scores do not correlate tovisual quality scores given by human raters [25].Christian E. Lopez, Scarlet S. Miller, and Conrad S. TuckerMD-18-1525Downloaded From: .org on 11/06/2018 Terms of Use: http://www.asme.org/about-asme/terms-of-use5
Journal of Mechanical Design. Received June 30, 2018;Accepted manuscript posted November 01, 2018. doi:10.1115/1.4041857Copyright (c) 2018 byASMEJournalof Mechanical Design Special Issue: Selected Papers from IDETC 2018.2.2 Crowdsourcing and generative design validationAs a result of the current limitations in the evaluation metrics of generative models,researchers are starting to integrate crowdsourcing methods to evaluate their models.editedFor example, Burnap et al. [3] use a crowdsourcing method to recruit 69 participantsand assess the ability of their deep generative model to generate realistic designs. Theirpyresults show that their model was able to generate realistic designs while exploring theCodesign space. Chen et al. [20] conduct a Turing test to compare the capability of 61human raters and four deep learning models to distinguish between human andotcomputer generated sketches. Their results reveal that some of the deep learningtNmodels outperformed the human raters in accurately distinguishing between humanscripand computer generated sketches. Dering and Tucker [4] use 252 human raters toevaluate the capability of their method to generate new 2D sketches that wereManurecognized to belong to a specific class. Their results indicate that human raters wereable to accurately recognize the sketches of certain classes. These studies have analyzedthe accuracy of human raters in classifying new images and sketches into specificedclasses, and not necessarily evaluating the functionality of sketches themselves.ptResearch indicates that crowdsourcing methods might constitute a promisingceparadigm for the product design process [26]. Table 1 shows a summary of existingAcliterature related to deep generative design tools and the implementation ofcrowdsourcing methods used to evaluate them. Most of the current works focus onevaluating the capability of deep generative models to create new sketches that can beclassified as belonging to a specific category. Though an idea needs to be new and novelChristian E. Lopez, Scarlet S. Miller, and Conrad S. TuckerMD-18-1525Downloaded From: .org on 11/06/2018 Terms of Use: http://www.asme.org/about-asme/terms-of-use6
Journal of Mechanical Design. Received June 30, 2018;Accepted manuscript posted November 01, 2018. doi:10.1115/1.4041857Copyright (c) 2018 byASMEJournalof Mechanical Design Special Issue: Selected Papers from IDETC 2018.in order to be considered creative, it also has to meet its intended functionality and beuseful [5].During the latter stages of the design process, designers create CAD models andeditedimplement advanced numerical methods to evaluate the functionality of their designideas. However, these methods are time-consuming and complex to implement, whichlimits their scalability [27]. Because of these limitations, researchers have started topyexplore how deep learning algorithms can be implemented to predict the ability of a 3DCoartifact to perform a function [28]. Nonetheless, during the early stages of the designotprocess, detailed 3D models are not widely available, compared to rough 2D sketches.tNSketches are typically the primary communication source of ideas, especially in the earlyscripphases of the design process [6,29]. Sketches can be categorized in terms of theirintended purpose, design progression, and physical elements [30–32]. Based on theirManuphysical elements, Rodger et al. [31] present categories ranging from simplemonochrome line drawings that do not include shading or annotations (Level 1), to highfidelity realistic sketches with extensive shading and annotations (Level 5). Severaledstudies have used these taxonomies to evaluate design sketches and explore how theyptare used in the early stages of the design process [33–35].ceRecently, researchers have started to integrate neural network algorithms andAccomputer simulation to predict the functionality of 2D sketches generated via deepgenerative design models. Cunningham and Tucker [36] present a Validation NeuralNetwork (VNN) that integrates a physics computer simulation. Frequently, during theinitial stages of the design process, designers use their experience and domainChristian E. Lopez, Scarlet S. Miller, and Conrad S. TuckerMD-18-1525Downloaded From: .org on 11/06/2018 Terms of Use: http://www.asme.org/about-asme/terms-of-use7
Journal of Mechanical Design. Received June 30, 2018;Accepted manuscript posted November 01, 2018. doi:10.1115/1.4041857Copyright (c) 2018 byASMEJournalof Mechanical Design Special Issue: Selected Papers from IDETC 2018.knowledge to ensure that generated ideas are relevant to the design problem. Forexample, experts have been used to evaluate and screen crowdsourced ideas [7,26].Similarly, crowds have been used to evaluate the perceptual attributes of new designsedited[37]. For instance, in the previous study of this work, the authors implement acrowdsourcing method to recruit 983 raters and explore the perceived functionality oflow-fidelity 2D sketches [7]. The results of the study reveal that participants perceivedpysketches generated via a deep generative model as more functional than human createdCosketches. Moreover, the results indicate that the perceived functionality of humanotgenerated sketches was negatively affected by explicitly presenting them as humantNgenerated sketches. Finally, the study reveals that participants were not able toscripaccurately distinguish between the human created sketches and the computergenerated ones.ManuWhile previous studies support the use of human raters to evaluate new ideas[26,37], the difference in the functionality evaluation of 2D sketch ideas between ratersand computer simulation has yet to be explored. If computational tools are to co-createednew products and solutions alongside designers, their capability to produce not onlyptnovel, but also functional ideas needs to be explored. Hence, in this work, the authorsceexpand on their previous study and explore the functional characteristics of 2D sketchesAccreated by humans and sketches generated via a deep generative design model, usingboth computer simulation and crowdsourcing methods.Christian E. Lopez, Scarlet S. Miller, and Conrad S. TuckerMD-18-1525Downloaded From: .org on 11/06/2018 Terms of Use: http://www.asme.org/about-asme/terms-of-use8
Journal of Mechanical Design. Received June 30, 2018;Accepted manuscript posted November 01, 2018. doi:10.1115/1.4041857Copyright (c) 2018 byASMEJournalof Mechanical Design Special Issue: Selected Papers from IDETC 2018.2.3 Designers’ biasesThe ability to generate creative ideas is an insufficient condition for innovationbecause decision-makers need to not only to generate, but also select creative ideas foreditedinnovation to occur [8]. Unfortunately, human bias can have a direct impact on thescreening and selection of ideas [38]. Studies indicate that decision-makers canpyexperience ownership [9], complexity [39], and even creativity biases [40]. SeveralCostudies have shown that the gender and risk attitudes of decision-makers can bias theirselection of ideas [9,41]. Similarly, the educational level and experience of individualsothas been related to their risk attitudes [11]. When evaluating the expertise of crowds,tNBurnap et al. [42] reveal that educational level and mechanical aptitude (e.g., domainscripknowledge) of raters was correlated to their capability to accurately evaluate designsolutions. Besides gender, educational level, and experience, age is another factor thatManucould affect designers’ decision-making when interacting with deep generative designtools. Studies have indicated that age can affect technology adoption, revealing thatyounger individuals value the usefulness of technology more than older individuals [15].edThis digital divide between generations is attributed to the fact that youngerptgenerations are exposed to digital technologies earlier in their life than oldercegenerations [43]. Moreover, studies indicate that technology acceptance and perceivedAcusability are affected by age [13,14].Besides decision-makers’ biases towards creative ideas, researchers have recognizedthat individuals can be biased towards automated systems (i.e., Automation bias)[44,45]. One of the factors that contribute to Automation bias is the trust given toChristian E. Lopez, Scarlet S. Miller, and Conrad S. TuckerMD-18-1525Downloaded From: .org on 11/06/2018 Terms of Use: http://www.asme.org/about-asme/terms-of-use9
Journal of Mechanical Design. Received June 30, 2018;Accepted manuscript posted November 01, 2018. doi:10.1115/1.4041857Copyright (c) 2018 byASMEJournalof Mechanical Design Special Issue: Selected Papers from IDETC 2018.automated support systems. This trust is the product of humans’ perception of thesesystems as having superior analytical capabilities than their human counterpart [46]. Forexample, the results by Dzindolet et al. [47] indicate that participants expected aneditedautomated support system to outperform the human system in a visual detection task.Studies on Automation bias focus on safety and automation aids, and not directly ondecision-makers’ biases towards early stage conceptual design tools. Hence, aspydesigners are increasingly integrating computational tools into the design process, theirCopossible biases towards computer generated ideas, compared to human created ideas,otneed to be explored. Also, more research is needed to understand the possible biasestNand the effects that individuals’ demographic characteristics and domain knowledgescriphave on their perceived functionality of 2D design sketches.In light of existing knowledge gaps, this work implements computer simulation andManucrowdsourcing methods to explore the functional characteristics of 2D design sketchesgenerated via a deep learning generative model, compared to human created sketches.The computer simulation enables the virtual physics-based evaluation of sketches toedperform their intended function. The crowdsourcing method enables the evaluation ofptthe perceived functionality (i.e., perception of how likely design sketches will perform acegiven function) of computer generated sketches, compared to the perceivedAcfunctionality of human created sketches. As a result, the possible effects of individuals’age, gender, educational level, and domain knowledge on their perceived functionalityare quantified. Moreover, the integration of computational simulation andcrowdsourcing methods allows for the comparison of the functional characteristics ofChristian E. Lopez, Scarlet S. Miller, and Conrad S. TuckerMD-18-1525Downloaded From: .org on 11/06/2018 Terms of Use: http://www.asme.org/about-asme/terms-of-use10
Journal of Mechanical Design. Received June 30, 2018;Accepted manuscript posted November 01, 2018. doi:10.1115/1.4041857Copyright (c) 2018 byASMEJournalof Mechanical Design Special Issue: Selected Papers from IDETC 2018.the sketches against their perceived functionality. In this work, the term ‘sketch’ is usedto mean a low-fidelity, rough 2D drawing representation of an idea with no shading oreditedannotations.3. RESEARCH QUESTIONSThis work aims to test the following hypotheses and address research questionspy(RQ):CoRQ1: Do individuals’ gender, age, educational level, or domain knowledge affecttheir perceived functionality of 2D computer and human generated sketches?tNotRQ2: Do individuals’ gender, age, educational level or domain knowledge affect theirbias towards the perceived functionality of computer or human generated sketches (i.e.,scriplabeling effect)?RQ3: Does the functional evaluation of computer simulation correlate to humans’Manuperceived functionality of 2D human and computer generated sketches?The authors hypothesize that (h1): individuals’ perceived functionality of 2Dedcomputer and human generated sketches is correlated with their age, gender,pteducational level, and domain knowledge. The authors hypothesize that the perceivedcefunctionality of male raters is different from those of female raters. In addition, theyAchypothesize that raters’ perceived functionality will be positively corrected to their age,educational level, and domain knowledge. These hypotheses are grounded in researchthat reveals that individuals’ demographic characteristics and domain knowledge levelcan relate to their decision-making process, technology adaptation, and evaluation ofChristian E. Lopez, Scarlet S. Miller, and Conrad S. TuckerMD-18-1525Downloaded From: .org on 11/06/2018 Terms of Use: http://www.asme.org/about-asme/terms-of-use11
Journal of Mechanical Design. Received June 30, 2018;Accepted manuscript posted November 01, 2018. doi:10.1115/1.4041857Copyright (c) 2018 byASMEJournalof Mechanical Design Special Issue: Selected Papers from IDETC 2018.design ideas [9,11,15,42] (see section 2.3). Testing this hypothesis will allow the authorsto address RQ1. The hypothesis can be mathematically expressed as:For,edited𝑃𝐹 𝛽0 𝛽1 (𝐺 ) 𝛽2 (𝐴𝑔𝑒) 𝛽3 (𝐺𝑒𝑛𝑑𝑒𝑟) 𝛽4 (𝐸𝑑𝑢𝐿) 𝛽5 (𝐷𝑘) 𝜀(h1) ho: 𝛽𝑖 0 vs. ha: 𝛽𝑖 0 𝑓𝑜𝑟 𝑖 ϵ {1 5}Where,(1)𝑃𝐹 is the individual’s perceived functionality of 2D sketches. 𝛽1 is the coefficient terms for the categorical variable for either computer generatedCopy otor human generated.𝛽2 is the coefficient terms for the variable of the individual’s age. 𝛽3 is the coefficient terms for the variable of the individual’s gender. 𝛽4 is the coefficient terms for the variable of the individual’s educational level. 𝛽5 is the coefficient terms for the variable of the individual’s domain knowledge.ManuscriptN Moreover, following RQ2, the authors hypothesize that (h2): individuals’ bias towardsthe perceived functionality of 2D sketches is correlated with their age, gender,ededucational level, and domain knowledge. That is, these factors will confound the effectsptof explicitly presenting the 2D sketches as computer generated or human created on theceindividual’s perceived functionality (i.e., with a label as in Fig. 1). The authorsAchypothesize that raters’ bias towards the perceived functionality of the sketches willdiffer based on their gender, age, educational level, and domain knowledge. Thishypothesis is grounded in research that reveals that decision-makers’ biases areChristian E. Lopez, Scarlet S. Miller, and Conrad S. TuckerMD-18-1525Downloaded From: .org on 11/06/2018 Terms of Use: http://www.asme.org/about-asme/terms-of-use12
Journal of Mechanical Design. Received June 30, 2018;Accepted manuscript posted November 01, 2018. doi:10.1115/1.4041857Copyright (c) 2018 byASMEJournalof Mechanical Design Special Issue: Selected Papers from IDETC 2018.correlated with their demographic characteristics and experience level [10,40,41], and itis expressed as:For,edited̅̅̅̅ ) 𝛽3 (𝐴𝑔𝑒) 𝛽4 (𝐺𝑒𝑛𝑑𝑒𝑟) 𝛽5 (𝐸𝑑𝑢𝐿) 𝛽6 (𝐷𝑘) 𝜀𝑃𝐹 𝛽0 𝛽1 (𝐺 ) 𝛽2 (𝑃𝐹(h2) ho: 𝛽𝑖 0 vs. ha: 𝛽𝑖 0 𝑓𝑜𝑟 𝑖 ϵ {1 6}Where,(2)Coeither computer or human generated (i.e., with a label).py 𝑃𝐹 is the individual’s perceived functionality of 2D sketches explicitly presented asot 𝛽1 is the coefficient terms for the categorical variable for either computer generatedtNor human generated.presented without labels.scrip 𝛽2 is the coefficient terms for the average perceived functionality of 2D sketchesManu 𝛽3 is the coefficient terms for the variable of the individual’s age. 𝛽4 is the coefficient terms for the variable of the individual’s gender. 𝛽5 is the coefficient terms for the variable of the individual’s educational level.ed 𝛽6 is the coefficient terms for the variable of the individual’s domain knowledge.ptFinally, the authors hypothesize that (h3): individuals’ perceived functionality of 2Dcesketches is positively correlated with the functional evaluation of a computer simulationAcof the same sketches. This hypothesis is motivated by studies that indicate the benefitsof using human raters to evaluate and select crowdsourced ideas [26,48]. Testing thishypothesis will enable the authors to address RQ3. The hypothesis is expressed as:Christian E. Lopez, Scarlet S. Miller, and Conrad S. TuckerMD-18-1525Downloaded From: .org on 11/06/2018 Terms of Use: http://www.asme.org/about-asme/terms-of-use13
Journal of Mechanical Design. Received June 30, 2018;Accepted manuscript posted November 01, 2018. doi:10.1115/1.4041857Copyright (c) 2018 byASMEJournalof Mechanical Design Special Issue: Selected Papers from IDETC 2018.𝑔𝑔(h3) ho: ρ̅̅̅̅ 0 vs. ha: ρ̅̅̅̅ 0 𝑔 ϵ {computer generated,PF,CSPF,CShuman generated}Where,edited̅̅̅̅ is the average perceived functionality of 2D sketches. 𝑃𝐹 𝐶𝑆 is the computer simulation’s evaluation of the 2D sketches functionality.py4. CASE STUDYCoTo address the previous research questions and test the hypotheses, a case study inwhich 2D boat sketches generated by humans and a deep generative model wereotpresented to raters recruited via a crowdsourcing platform and evaluated using a physics4.1 Dataset of 2D sketchesscriptNcomputer simulation.ManuFor this case study, the Quick, Draw! dataset was utilized [49]. This dataset wasacquired by Google via the Quick, Draw! game. In this game, individuals are asked toeddraw a specific object within 20 seconds (e.g., “draw a boat in under 20 seconds”). Forptthis case study, a total of 132,270 human created boat sketches were used as a trainingcedataset for the Sketch-RNN algorithm [19]. The model generated by the Sketch-RNNAcalgorithm (see Ha and Eck’s [19]) was used to generate 250 new boat sketches. Fromthese 2D boat sketch datasets, 50 computer and 50 human sketches were randomlyselected for evaluation. Figure 1 show some of the human and computer generated boatsketches used.Christian E. Lopez, Scarlet S. Miller, and Conrad S. TuckerMD-18-1525Downloaded From: .org on 11/06/2018 Terms of Use: http://www.asme.org/about-asme/terms-of-use14
Journal of Mechanical Design. Received June 30, 2018;Accepted manuscript posted November 01, 2018. doi:10.1115/1.4041857Copyright (c) 2018 byASMEJournalof Mechanical Design Special Issue: Selected Papers from IDETC 2018.4.2 CrowdsourcingIn this work, Amazon Mechanical Turk (AMT) was used as the crowdsourcingplatform to recruit raters. AMT has been previously used to evaluate the output of deepeditedgenerative models [3,4]. Moreover, AMT has established itself as a valuable tool forbehavioral research since studies have found no significant differences in the responsepyconsistency between internet users and laboratory participants [50,51]. Compared toCoother crowdsourcing platforms, AMT provides the benefits of (i) low cost, (ii) large raterpool access, and (iii) large rater pool diversity [51]. In this work, a total of 1,187 ratersotwere recruited to evaluate a set of boat sketches, which expand the number oftNparticipants from the previous study by 204 individuals [7]. The raters werescripcompensated 0.20 for their participation in the experiment. Only raters with a 90%satisfaction rate were allowed to participate in this experiment. Similarly, participantsManuwere only allowed to take the quest
Journal of Mechanical Design Special Issue: Selected Papers from IDETC 2018. Christian E. Lopez, Scarlet S. Miller, and Conrad S. Tucker MD-18-1525 2 that both participants and computer simulatio
The Scarlett 2i2 has a single USB 2.0 port (on the rear panel). Once the software installation (if necessary) is complete, simply connect the Scarlett 2i2 to your computer using the USB cable provided. Note that Scarlett 2i2 is a USB 2.0 device, and thus the USB connection requires a USB 2.0 compliant port on your computer.
IMPORTANT: The Scarlett Solo has a single USB 2.0 Type C port (on the rear panel): connect it to your computer using the USB cable provided. Note that Scarlett Solo is a USB 2.0 device, and thus the USB connection requires a U
MILLER Syncrowave 350LX TIG Welder, 350-AMP @ 40% w/Miller Cooler MILLER Shopmaster TIG Welder, 300-AMP @ 50% w/Miller Cooler MILLER Dialarc HF TIG Welder, 250-AMP @ 40%, Miller Coolmatic 3 MILLER DX Portable TIG Welder (11) MILLER Dialarc 250 AC/DC Stick Welders, 225-AMP @ 30% (3) LINCOLN Idealarc 250 Stick
(10) Late Model Miller and Lincoln Tig Welders; (2) Lincoln Precision Tig 375, (2) Miller Dynasty 350’s, (2) Miller Syncrowave 350LX’s, Miller Dynasty 300DX, Miller Syncrowave 250, (2) Miller Dynasty 200’s SEE PHOTO (8) Miller Spectrum 2050 Plasma Cutters SEE PHOTO Large Quantity of W
111. Burt Bacharach 4. All That Heaven Allows Rep. Manuscript Away All Boats Rep. Manuscript Swell Guy Rep. Manuscript Star in the Dust Rep. Manuscript Shenandoah Rep. Manuscript Music by Frank Skinner Universal Studios 4-78RPM Back Street Arabian Nights Saboteur Back Street (A Collection) Universal Studios Universal Studios
350 Amp Miller Millermatic 350P Pulsed Mig Welder 300 Amp Miller Millermatic 300 Mig Welder (13) 250 Amp Miller Millermatic 252 Mig Welders (6) 250 Amp Miller Millermatic 251 Mig Welders (3) 250 Amp Miller Millermatic 250 Mig Welders 250 Amp Miller Millermatic 250X Mig Welder (4) Mi
(2) 350 AMP Miller Millermatic 350P (Aluminum) Mig Welder 350 AMP Miller Dynasty 350 Tig Welder 200 AMP Miller Dynasty 200 Tig Welders 200 AMP Miller Millermatic 252 Mig Welder 250 AMP Miller Millermatic 250 Mig Welder (2) 250 AMP Miller Syncrowave 250 Tig Welder 275 AMP Lincoln Precision Tig
Although adventure tourism is recognized as an important, growing tourism segment, primary research to quantify the size and scope of this market in the U.S. or internationally (Schneider 2006) has been lacking. For this reason, George Washington University, along with its partners, the Adventure Travel Trade Associ-ation (ATTA) and Xola Consulting, sought to better understand the adventure .
