Evaluating a new programming language

Keywords: POP-III.B. C # POP-III.C. all cognitive dimensions POP-V.B. questionnaire We describe our efforts using the Cognitive Dimensions framework to evaluate a new programming language. We used a questionnaire approach to gather framework data related t

In G. Kadoda (Ed). Proc. PPIG 13Pages 275-289

Evaluating a new programming language

Steven Clarke

Microsoft Corporation

1 Microsoft Way

Redmond, WA 98052 USA

+1 425 705 5978

stevencl@http://doc.guandang.net

Keywords: POP-III.B. C# POP-III.C. all cognitive dimensions

POP-V.B. questionnaire

Abstract

We describe our efforts using the Cognitive Dimensions framework to evaluate a new programminglanguage. We used a questionnaire approach to gather framework data related to the new languageand combined this with a traditional lab-based evaluation. With this approach, we were able tovalidate the findings from the cognitive dimensions questionnaire by correlating them with thefindings from the lab-based study. From this, we were able to determine if it is feasible to performsuch an evaluation with usability participants who have no prior experience of the language beingevaluated. Our findings have implications for anyone intending to use the cognitive dimensions toevaluate new notations or programming languages.

Introduction

To evaluate the usability of a new programming language, called C#, developed at Microsoft Corp.,we used the Cognitive Dimensions framework (Green and Petre, 1996). This paper describes ourexperiences using the framework to evaluate a new language for which no experienced users yetexist.

The paper starts by presenting a brief overview of the Cognitive Dimensions framework. Followingthis, the next section describes the study undertaken, followed by the analysis of the data gatheredfrom the study. Lastly, a discussion of the results of the study and a description of the further worksuggested by the study is presented.

Using the Cognitive Dimensions

Cognitive Dimensions is a framework for analysing the usability of a notation, or programminglanguage. The framework presents thirteen dimensions upon which judgments are made regardingcognitive demands made by a particular notation or programming language. Due to space constraints,the dimensions are not described here. Interested readers are referred to Green and Petre, (1996) formore detail.

Having evaluated a number of different methods for using the cognitive dimensions (Clarke andSpencer, 2000; Cox, 2000; Modugno et al, 1994; Roast and Siddiqi, 1996) we settled on the

questionnaire approach described in Blackwell and Green (2000). The questionnaire is neutral to thelanguage being evaluated and so we did not need to do any additional work to be able to apply it inour situation.

However, one issue with using the questionnaire was whether or not we would be able to use thissuccessfully with a set of participants who have no experience with the programming language beingevaluated. Another issue was validity. How could we determine the validity of the results obtainedfrom applying the cognitive dimensions in this way? In the next section, we describe the study wecarried out, and how we attempted to deal with participant’s lack of experience with the programminglanguage and the issue of validity.

Methods and Procedure

The study offered an opportunity to answer a number of research questions. First, we wanted todetermine how effectively we could use the cognitive dimensions questionnaire in cases whereparticipants have no experience with the notation or programming language being evaluated.

Secondly, we wanted to be able to assess and demonstrate the validity of this method of evaluation.

Keywords: POP-III.B. C # POP-III.C. all cognitive dimensions POP-V.B. questionnaire We describe our efforts using the Cognitive Dimensions framework to evaluate a new programming language. We used a questionnaire approach to gather framework data related t

And thirdly, we wanted to learn about the kinds of problems that users might experience whenprogramming with the new programming language, C#.

New Features of C#

Before describing the study in detail, it is worthwhile to describe those features of C# that we weremost interested in studying in a usability study.

C# is a new object oriented language, based on C++. While the syntax of the language is similar toC++ there are a number of new features:

Properties. These are standard get and set member variable functions that allow programmers

to write code that looks like they are directly accessing class member variables. This is

intended to make some code easier to read and write.

Versioning. Introduction of a new member in a base class with the same name as a member in

a derived class is not an error in C#, unlike other languages, such as C++. A class must

explicitly state whether a method is intended to override an inherited method, or whether amethod is a “new” method that simply hides a similarly named inherited method. This is

intended to make it easier for programmers to adapt to changing requirements.

No pointers. C# programmers do not manipulate memory directly. Unreferenced memory is

garbage collected automatically, and all references to objects are by-reference, rather than by-value.

Subjects

Five professional programmers participated in the study. Four had at least two years of

professional experience programming in C++. One participant was a novice, and had six monthsexperience working as a professional developer, having recently graduated from High School.Test Methods

In order to address both the issues of experience and validity, we decided to run a traditional

laboratory based usability evaluation, and to administer the cognitive dimensions questionnaire after …… 此处隐藏：30110字，全部文档内容请下载后查看。喜欢就下载吧 ……