Evaluating Open Source Software Usability Using A .

Thamer A. Al-RawashdehThis leads to the usability characteristics cannot bemeasured directly, but it should be evaluated in terms ofcharacteristics objective and software [17]. This includes,the context of evaluation should be taken intoconsideration, before determining on specific usabilitycharacteristics to be used [21], [22].In the same regard, according to Benson et al., 2004[27] the OSS usability related issues are not the same asthose for closed proprietary software. Therefore, theproposed fuzzy usability model describes the features; ofOpen Source Software and the usability characteristicsthat are related to such features.III. Characteristics AffectingOOS acteristics that are general and common forevaluating the usability of every type of softwareproduct. However, each software type has its owncharacteristics [19]. Therefore, the existing usabilitycharacteristics should be modified and extendedaccording to the nature of a new software beingevaluated. This includes, redefining some characteristics,eliminating others, and introducing new usabilitycharacteristics.Therefore, totally nine usability characteristics havebeen refined and introduced in this section according tothe Open Source Software nature and its features. Eightusability characteristics are adapting: learnability,understandability, efficiency, memorability, errorprevention, operability, attractiveness, and usabilitycompliance; and new one has been introduced namelysoftware familiarity.Learnability is defined as the capability of software toenable users to learn the main software functionality andto obtain proficiency to complete a task [23]. In OpenSource Software context, the learnability could beevaluated by measuring the time a user needs to performa task in the time an expert would take to complete thesame task.Understandability refers to the capability of asoftware product to enable the user to understandwhether the software is suitable, and how it can be usedfor particular tasks and conditions of use [24]. In thisrespect, in addition to the purpose of Open SourceSoftware, all of the design and user documentations mustbe clearly written, so that it is easy to understand.Efficiency is described as the number of tasks that auser can perform using a software per unit [25], [26]. Itcould be measured by the task completion rates and taskcompletion time. The more efficient Open SourceSoftware is the faster users can perform a task andcomplete a task. This is among the major characteristics,as a reason of this concept gaining popularity.Memorability is related to the capability of a softwareto enable users to use the software when they return to it[14]. The importance of memorability for Open SourceSoftware is some Open Source Software i.e. operatingCopyright 2015 Praise Worthy Prize S.r.l. - All rights reservedsystems (Linux) are quite complicated, so the highmemorability software enables users to use suchapplications when they come back to them. Thememorability characteristic is influenced by varietyfactors. For instance, an action makes the users have areaction, if this reaction leaves the users with a goodfeeling they will not forget it.The Errors Prevention characteristic refers to thenumber of errors a user makes while performing a taskrather than the software [27]. Low error rate while OpenSource Software being used implies a good usability. Onthe contrary, high error rates, a low efficiency andsatisfaction [28]. In the same respect, proper design oferror message is an important topic in software usability.Therefore, many factors should be taken into accountto make such error messages helpful to the users. Amongthese factors are the error message should exist, themessage should understandable, the message should beprecise, the message should be correct, and the messagesshould be consistent. Furthermore, error preventionimprovises time consumption while the developmentprocesses [29].The Operability characteristic is related to thecapability of software to enable the user to operate andcontrol it [14]. There are many factors that affect thesoftware operability: suitability of the softwarecomponents for a task, self-descriptiveness of thesoftware components, the controllability of the softwarecomponents, the conformity of the software componentswith users’ needs, error tolerance of the softwarecomponents, and suitability of the software componentsfor individualization [14]. Therefore, all of these factorsshould be considered while OSS is being developed.Attractiveness refers to the capability of software to beattractive to its users [30]. According to Maedche eat al.,2012 [30] the software attractiveness has been influencedby pragmatic and hedonic factors perspicuity, up-todatedness, aesthetic, simulation, and novelty.Thus, the user interface of Open Source Softwareincluding windows, menus, bars, messages etc. shouldbe consistent, aesthetic and visually attractive.Additionally, it is worth to mention that, due to theaccessibility of Open Source Software code, users wouldhave an ability to change the user interface componentsaccording to their desires.Usability compliance is defined as the capability of asoftware to adhere to standers, conventions, style guide,or usability regulations [14]. The evaluation of OSSusability compliance with ergonomic standards cannotdetermine the quality of the OSS, but it determineswhether the OSS meets the usability requirements andrecommendations that found in various ergonomicstandards, laws, or/ and style guides. Among suchstandards are ISO 9241, ISO 14915, ISO 16071, ISO23973, ISO/IEC 11581, ISO/IEC 18035 & 18036, ANSICIF, and HFES/ANSI 200 [47].Familiarity is another factor that affects the softwareusability [11]. According to Seffah eat al., 2006 [31] thefamiliarity refers to the capability of the software’sInternational Review on Computers and Software, Vol. 10, N. 101020

Thamer A. Al-Rawashdehinterface to provide recognizable elements that can beunderstood by the user. In the user interface context,most of the computer users are familiar with the closedproprietary software such as Mac, Windows, andMicrosoft internet explorer etc.Therefore, in order to improve its usability, the userinterface of Open Source Software should be designed insome way to be like components of common proprietarysoftware. Furthermore, the familiarity could be obtainedby information that provided by external resources suchas advertising and word-of-mouth [32].IV. Multistage Fuzzy Model Approachfor OSS UsabilityAlthough the literature of software usability providesmany ways and methods to evaluate software usability,there is no exact approach to do so. Further, evaluation ofsoftware usability is depend on various characteristics,which are fuzzy in nature. Therefore, a new multistagefuzzy model approach for evaluating usability of OSS ispresented in this paper. The advantage of using fuzzylogic is that it provides a convenient way to transformknowledge expressed in a nature language into fuzzylogic rules [31]. Multistage Fuzzy Model approachassists in decision making and control in Fuzzy-basedtechniques. It should be noted that, fuzzy basedapproaches divided into two approaches, including fuzzytraditional approach and the fuzzy reasoning approach[31]. In the next sections, a motivation of using the fuzzyreasoning approach in this paper is explained.IV.1. Fuzzy Traditional Approachpossible, since the number of inputs is limited to two ineach stage. The first group includes the characteristicsthat reflect the capability of software to enable users toperform tasks. These characteristics are ility,attractiveness and familiarity [14], [30]. The secondgroup comprises two characteristics, namely efficiencyand error-prevention. It should be noted that, thesecharacteristics are related to the users’ capability toperform the tasks [25]-[27], [32].Finally, the third group involves one characteristicwhich is usability compliance [14]. This characteristic isrelated to the software components. Table II shows thegrouping of the grouping proposed model characteristics.Fig. 1. Traditional fuzzy model approach of OSS usabilityTABLE IIGROUPING OF CHARACTERISTICSGroupCritical ElementSoftware-capabilitylearnability, understandability, memorability,operability, attractiveness, and familiarityUser-capabilityEfficiency, and error preventionSoftware-components Usability complianceFuzzy traditional approach assesses software usabilityusing all inputs (usability characteristics) by simple ifthen-rules [31]. Fig. 1 presents the traditional fuzzyapproach block diagram of OSS usability.The model in Fig. 1 considers all the characteristics asinputs, In order to rank the OOS usability. This leads togenerate too many rules. It is so difficult for experts toproperly consider and formulate such rules [33]. Since inthe simplest case, each input has three linguistic values(High, Moderate, and Low). Hence, OSS with nineusability characteristics will have a maximum number of39 19683 rules. A MATLAB-Fuzzy Tool Box could notbe used to implement such fuzzy model approach.Since a number of inputs in the MATLAB is limitedto two. So the solution of this problem lies in using stagewise fuzzy reasoning approach which reduces thenumber of rules by dividing software into several fuzzyinterface stages [6], [34] and effectively measure theusability of Open Source Software.Hence, OOS usability can be divided into multiplethought process. Learnability and understandability areused in fuzzy reasoning to determine intermediateparameter (UA1), Memorability and operability todetermine another intermediate parameter (UA2),attractiveness and Familiarity to determine (UA3).In the same way, the characteristics of the secondgroup (efficiency and error-prevention) are used thefuzzy reasoning to determine new intermediate parameter(T) as shown in Fig. 2(a), which is later combined withthe third group characteristic (usability-compliance) todetermine new output parameter namely UA5.Consequently, both UA3 and AU5 are used togenerate new output parameter (UAT) see Fig. 2(b).Similarly, the output parameters UA1 and UA2 arecombined to generate UA4 as new intermediateparameter which is later combined with UAT to get finaloutput parameter (OSSU) Fig. 2(c).IV.2. Fuzzy Reasoning ApproachIV.2.1. FuzzificationThe proposed model is based on dividing the usabilitycharacteristics into three groups. Using stage wise fuzzyreasoning approach will also make MATLAB usageThe Fuzzification process converts crisp input valuesinto the degree of membership using membershipfunction.Copyright 2015 Praise Worthy Prize S.r.l. - All rights reservedInternational Review on Computers and Software, Vol. 10, N. 101021

Thamer A. Al-RawashdehWhere, the membership function is defined as a curvethat defines each point in the input space is mapped to adegree of membership of fuzzy set [35]. In the proposedmodel, linguistic terms have been used to represent allinputs and outputs. These linguistic terms are representedby membership functions. In this study the triangulartype membership function has been used to model themembership degree of all input and output variables [37].The range of input variables including learnability,understandability, efficiency, memorability, operability,error-prevention, attractiveness, familiarity, and usabilitycompliance are represented as High, Moderate, and Low.In order to have a high accurate results, five membershipfunctions Very High, High, Moderate, and Low havebeen used for the output parameters as shown in Fig. 3.Fig. 3. Fuzzified output parametersIV.2.2. Approximate ReasoningAn approximate reasoning is a process that formulatedfor inference to deal with ambiguous cases. It involvessub-process such as fuzzy control rules and fuzzyinference engine [31]. Fuzzy rules are linguistic IFTHEN constructions that are used to represent the humanknowledge. These constructions have the general form ofIF premise THEN consequence [38]. The usage of fuzzyrules exploits the tolerance of imprecision anduncertainty. This will improve the ability of human tosummarize data and focus on decision-relevantinformation. In this study, a survey of 10 expertsincluding academics, software developers, researcherscholars, and OSS usability experts has been done tofinalize the fuzzy control rules. Fig. 4 shows the fuzzyrules of FIS1.Fig. 2(a). Learnability and usability fuzzy reasoningFig. 2(b). AU5 and AU3 fuzzy reasoningFig. 4. Fuzzy control rules of output erstandability are the input parameters of first FISand UA1 is gotten as an output parameter. Additionally,the fuzzy linguistic values of High, Moderate, and Loware assigned for the input parameters. Whereas, VeryHigh, High, Moderate and Low are some of the outputparameters. As seen from Figure above sixteen ruleshave been derived from the collected knowledge for theoutput of first FIS. E.g. if the rating for learnability ishigh and low for understandability then the overall ratingof the output parameter will be moderate. Similarly, theFig. 2(c). UAT and usability UA4 reasoningCopyright 2015 Praise Worthy Prize S.r.l. - All rights reservedInternational Review on Computers and Software, Vol. 10, N. 101022

Thamer A. Al-Rawashdehrules of rest eight FISs have been framed.V.Similarly, the rating of UA1 and UA2 have beencombined to rate UA4 as 6.13, using the fuzzy ruleswhich were determined in fuzzification stage. Later, theUA3 and UA5 ratings are combined to rate UATparameter as 6.13. Finally, the ratings of UAT and UA4passed to the OSSU controller in order to generate therating of overall OSS usability (6.14), then this rating hasbeen converted to a crisp value using the MATLABfunction (6).Simulation of Multi Stage FuzzyApproach Model for OSS UsabilityThe present work has been done, in order to propose anew multistage fuzzy approach model to evaluate theusability of Open Source Software. This proposed modelallows to pass the fuzzy control rules to another as factthrough intermediate variables (the output parameters).In this section, the MATLAB Simulink Software isused to simulate and link all proposed model stages. Inthe other words, the fuzzy logic controller is used to linkall FISs through passing the value of intermediatevariables to another as shown in Fig. 5. To validate theproposed multistage fuzzy model approach, a simulationhas been done using different sets of critical elementrating see Table III. Considering the scenario in theprevious table, the proposed multistage fuzzy modelapproach has rated the UA1, UA2, AU3, and T as 6.14,6.14, 8.62, and 3.63 respectively. The value of Tparameter has been passed and combined with the ratingof usability compliance to rate the output parameter(UA5) as 3.6.VI.Validation of Proposed ModelA standard Analytical Hierarchy Process Technique(AHP) has been applied to validate the proposed modeland to rank the usability characteristics. The AHP is amulti-criteria decision making method that was proposedby Thomas L. Saaty in 1980 [35], [36].A survey of twenty computer science graduatestudents has been done to collect data, which later usedwith the AHP technique to evaluate the Open SourceSoftware usability model and to rank the factors of suchmodel.All students have 2 to 3 years of experience usingOpen Source Software Linux, open office and other opensource applications.Fig. 5. Stage Wise Fuzzy Approach Model of OSS Usability-Simulink ModelCritical ncyError-preventionUsability compliance575777553TABLE IIISIMULATION DATAStage wise fuzzy model outputUA2 UA3TUA4 UA5 UATFinal RatingOSSU6.146.136.146.148.623.63Copyright 2015 Praise Worthy Prize S.r.l. - All rights reserved6HIGH6.133.6International Review on Computers and Software, Vol. 10, N. 101023

Thamer A. 81.001.370.260.200.190.55TABLE IVCHARACTERISTICS VALUES USING AHP .000.470.250.720.651.722.121.000.61Total12.70The survey consists of 36 comparison pair wisequestions to compare nine usability characteristics witheach other, in order to determine that which characteristicis more important and gets more priority for Open SourceSoftware.The mean of each question’s responses to the pairwiseweight values of the main usability characteristics arefilled in a square matrix Table IV.After getting the pairwise relative weight values of theusability characteristics, the nth root has been calculatedby multiplying all row values and then taking the (1/9)throot of each usability characteristic [24].As also mentioned in Table IV the nth values of theusability characteristics model are, for learnability (3.47),for understandability (3.05), for efficiency (2.05),memorability (1.11), for error-prevention (1.16), foroperability (0.64), for attractiveness (0.37), for usabilitycompliance (0.25), and for familiarity (0.61). The sum ofthese values is 12.7. Next Eigen Vector ( ) has beencalculated by dividing the nth root value of eachcharacteristic by the sum of all nth root values of thecharacteristics [39]. It could be seen that, the eigenvectorof relative importance or in the other words the weightsof the main usability characteristics of Open SourceSoftware from the students’ perspective as the following:learnability (0.27), understandability (0.24), efficiency(0.16), error prevention (0.091), memorability (0.087),operability (0.05), familiarity (0.048), attractiveness(0.029), and usability compliance (0.019).It is worth tomention that, the summation of eigenvector values is 1.0,hence the comparison values for the usabilitycharacteristics are right [19].Thus, from students’ perspectives learnability of theOSS is the most important one over other usabilitycharacteristics, followed by understandability, efficiency,error-prevention, memorability, portability, familiarity,attractiveness, and finally the usability compliance is lessimportant [40]. These results provide more than ranking.In fact, the relative weights are a ratio scale could bedivided among such usability characteristics.Human judgment is not always consistent, thus inorder to check the consistency of participants’ answers,the Consistency Index (CI) and Consistency Ratio (CR)must be obtained, Eqs. (2) and (3).To do so the eigenvalue is required, Eq. (1) [39]:A λmax , and n(1)Copyright 2015 Praise Worthy Prize S.r.l. - All rights 469.179.94where eigenvalue; and n is represents the number ofelements to be compared. The difference between andn is an indication of judgment consistency:CI max nn 1(2)CIRI(3)CR where CR is the average of consistency index and shouldsatisfy the condition of CR 0.1.For this study, all the values of the eigenvalue havebeen mentioned in the Table IV. These values reveal that value has satisfied the condition of n, (9.94 9).Consequently, by applying Eqs. (2) and (3), CI (9.94– 9) / 8 0.1175 and CR 0.1175/1.45 0.081, is beingcontent with the condition of CR 0.1 [39].VII.Contribution to the StudyFuzzy logic allows infinite intermediate logical valuesbetween the false and true. Another study in this regardrepresents mapping of analytical tools for modeling andcomputational treatment of Fuzzy Systems.This mapping showed the existence of relationshipsand the appropriateness of using the Fuzzy Theory inbuilding models for solving problems involvingdynamical systems and, in particular, of ChaoticDynamical Systems [33]. It was proposed to differentiatebetween two types of Dynamic Fuzzy Systems that isIntrinsic Fuzzy Dynamical Systems and DynamicalSystems Fuzzy Extrinsic. Dynamic systems were used inFuzzy modeling for trajectories of fixing problems inchaotic systems.Case studies were developed that allowed to verifythrough simulations and tests in billiards Chua circuit(the latter implemented in physical prototype) theappropriateness of using this technique in solving theseproblems. Similarly, the qualitative research presentedhere aims to discuss the value and role of fuzzy logic insolving real problems, given the characteristics of itstools to deal with subjective questions, since thetroubleshooting from the real world a

multistage fuzzy model approach to evaluate and quantify the usability of OSS [6]. This includes factors that should be considered when designing and implementing an OSS. It gathers three main contributions including: the conceptual model that provides requirements for developing an OSS with desired usability; adapting and