Leave a comment

Understanding Build Server and Configuration Management – Part 1

A two-part series by Morgan McCollough, Senior Software Engineer

Morgan McCollough Bridge360The terms configuration management, continuous integration, and build server are used like buzzwords in the software world today. But how many people really understand what they mean in terms of time, efficiency and money? Most computer science programs in universities don’t talk about the practical application and benefit of these concepts, possibly because they are beyond the purview of the academic science.

In general, software configuration management is the systematic management of software changes both in terms of code and configuration for the purpose of maintaining integrity and traceability throughout the life cycle. In practical terms this can mean a wide variety of things, all the way from a simple source control branching and release management process to CMM level 5.

Continuous integration is the practice of merging all developer working copies of a source tree several times a day into a central development branch for the purpose of building, testing, and providing instant feedback. Implementing some version of one or both of these environments requires the creation of a centralized build server, the system where all code is merged, builds are produced, unit tests are executed, and eventually automated deployments are initiated.

In my line of work, I am in the position to work with a number of different clients and see diverse development environments and teams. In teams larger than two people, the difference between those that use continuous integration and solid configuration management and those that don’t is significant. I think it is very likely that many teams don’t implement continuous integration or a build server because they don’t fully appreciate what can really go wrong or the potential negative effects when they don’t have these systems in place. The time wasted doing builds, fixing deployment configurations, and correcting broken builds when configuration management is not in place is a true hidden cost in software development.

Teams that have these systems in place:

  • Have a more stable code base
  • Tend to be more flexible
  • Can deploy and test in disparate and new environments with minimal effort, and
  • Can reproduce any specific build or release from any point in the history of the application

Many software development teams are familiar enough with the concepts of build servers, configuration management and continuous integration to know they are a good idea, but often decide it’s more trouble than it is worth to implement them. In my next post, I’ll outline the basic steps you can follow to implement continuous integration and configuration management in your development environment.


Leave a comment

The Convenience of ClojureScript for Complex Applications

by Paul Bostrom, Senior Software Engineer

Paul-BostromIn my last blog post, The Case for Clojure, I discussed the advantages of using the Clojure programming language for writing applications on the Java Virtual Machine (JVM). Clojure’s success on the JVM led the language designers to compile the language for JavaScript engines, resulting in ClojureScript. Practically every device that can connect to the Internet has a web browser containing a JavaScript engine, making it an appealing target for developing applications.

Unfortunately, the JavaScript language contains flaws that make it difficult for developers to manage the complexity inherent in large applications. The widespread adoption of JavaScript presents a barrier to fixing these flaws without breaking existing applications. The JavaScript community has constructed various conventions in attempt to refine JavaScript’s faults, however these conventions have an ad-hoc nature and are not consistently applied. ClojureScript provides developers with an entirely new language and set of semantics that can be executed on any existing JavaScript engine. ClojureScript avoids JavaScript’s shortcomings while providing a number of new programming paradigms not currently available in the JavaScript ecosystem.

A Comparison of Clojure and ClojureScript

Clojure and ClojureScript are essentially identical languages, although they differ in the way that they deal with their respective execution environments, the JVM and the JavaScript engine. ClojureScript has the ability to interoperate with all of the interfaces that JavaScript uses, like the browser window, the document object model (DOM), and HTML5 APIs. ClojureScript uses the same immutable data structures as Clojure on the JVM. In a multi-threaded environment like the JVM, immutable data structures provide a nice alternative to the complex locking code used to preserve the integrity of data structures across multiple threads. Almost all JavaScript engines are single threaded, and only one particular section of code can run at any time. However, JavaScript engines are designed to handle asynchronous events that might be generated by a user interaction like a mouse click. It is hard to predict in what order related sections of code will run when developing JavaScript applications since these events can occur at any time. Immutable data structures make the development process easier by explicitly delineating when application data can be modified.

ClojureScript and JavaScript each share some common characteristics, so the difference between program structures is not as vast as the difference between Clojure and Java. ClojureScript and JavaScript are both dynamic, functional languages, so many concepts can be expressed with the same degree of conciseness. ClojureScript has an additional advantage over JavaScript, though, since it uses the same macro system provided by Clojure. This allows developers to extend the capabilities of the compiler. Well-designed macros allow repetitive, boilerplate code to be abstracted away, resulting in much more concise source code.

Namespaces

ClojureScript enforces a specific organizational structure for code called namespaces. All libraries outside of the core language must be referenced by their unique namespace (similar to Java packages). The lack of an equivalent structure can be a pain point for JavaScript developers. An interesting scenario involves the use of the popular JavaScript libraries jQuery and Prototype. Each of these libraries defines a global symbol ‘$’. JavaScript code using both of these libraries cannot differentiate between calls to ‘$’ in jQuery or Prototype without writing additional code to manage this symbol conflict. ClojureScript’s namespaces conveniently avoid such conflicts.

Development Tools

The ClojureScript community has contributed a number of tools to increase productivity during application development. The tool ‘cljsbuild’ leverages the Clojure build tool ‘lein’ to automatically download third-party dependencies and then compile ClojureScript code into JavaScript. It has a convenient feature to automatically recompile when it detects changes to the source code.

One of the most innovative development tools for ClojureScript is called the “browser REPL.” The acronym “REPL” stands for “Read-Evaluate-Print-Loop.” It is a concept that is found in a number of development languages and usually consists of a text prompt that accepts source code to be instantly evaluated. The browser REPL is an extension of this idea and will evaluate ClojureScript code in the context of a running web browser, skipping the usual compile/refresh process. While this is certainly a valuable tool, most major web browsers provide a similar console for evaluating JavaScript code. The ClojureScript community has taken this concept a step further and integrated the browser REPL with a number of popular text editors. This allows editing of currently executing ClojureScript source code directly in the browser from the comforts of a full-featured text editor. There are few other development environments that offer such a dynamic experience.

Language Innovation

The Clojure development community has a reputation for producing innovative new libraries to resolve issues across all types of problem domains. A recent example of this is a new library to do asynchronous programming called core.async. The main architect of the library (as well as the creator of Clojure) describes the library here. To summarize, JavaScript applications have traditionally used callback functions to handle asynchronous events like mouse clicks or responses from AJAX calls. This library has the potential to simplify how ClojureScript applications are structured, providing a nice alternative to callback functions. Due to the compiled nature of ClojureScript, in addition to the macro system’s ability to create abstractions out of portions of code, a library like core.async can provide new syntax without introducing any drastic changes to the language or the runtime environment. In contrast, the standard that JavaScript is based on, called ECMAScript, moves at a much slower pace in order to standardize any new syntax. A lot of the features available today in ClojureScript cannot be introduced in the ECMAScript standard until a large group of stakeholders agree upon them. This is often necessary due to the importance of stability in future JavaScript engines, but it often leaves JavaScript developers wanting for more powerful language features for constructing applications.

As browser applications become more complex, JavaScript developers must expend additional effort to produce correct, maintainable code. ClojureScript provides numerous features for dealing with application complexity and creating clear and concise code. I encourage readers to consider using ClojureScript for your next web-based project.


2 Comments

The Case for Clojure

By Paul Bostrom, Senior Software Engineer

Paul-BostromChoosing a programming language for a project is difficult. As technology progresses, new languages emerge exponentially, each one with a particular set of features; others come along that seek to improve existing features, as well as solve new problems. Clojure is a relatively new language, released in 2007 by Rich Hickey. Clojure is a Lisp-like, functional language that runs on the Java Virtual Machine. Let’s look at some of the features that make Clojure a compelling choice.

The Java Virtual Machine

The Java programming language was introduced in 1995 and provided a powerful general purpose programming language. One particular feature it provided was automatic memory management and garbage collection, so that programmers did not have to write code to manage memory like in a C or C++ program. Java also provided an execution environment called the Java Virtual Machine (JVM) to allow programs to be portable across a number of different platforms. For these and other reasons, Java has grown to become one of the most popular programming languages.

The popularity of Java has made the JVM an ideal target for the development of newer programming languages like Clojure. Just as the Java language provided features not available in C or C++, Clojure attempts to provide additional features that Java may not include, while building upon the many features that are included in the Java programming environment. Clojure also provides a number of Java interoperability features to work with the many stable Java libraries in existence. What this means in practical terms is that Clojure code can exist in a variety of forms. Perhaps you just want to write a Clojure program that is simply “glue code” to integrate several reliable Java libraries into a single application. Another option is to create a specialized library in Clojure that Java programs can utilize. Of course, Clojure provides a rich core library that allows the creation of general purpose programs that do not require any direct interaction with existing Java code.

Conciseness

Clojure code is a lot shorter than equivalent code in Java. There are a number of reasons for this, but it’s important to note that the length of the code in a software program has a direct correlation with the cost of developing and maintaining the software. Suppose a Java program is 50 lines, while the equivalent Clojure program is 20 lines, which in my experience is a typical code reduction factor between Java and Clojure. The programmer will spend less time thinking about, writing, and testing the 20-line Clojure program than the 50-line Java program, even though both programs accomplish the same task. One of the reasons Clojure programs are so much shorter is because it has dynamic types, meaning the code doesn’t require a lot of extra statements to declare the various constructs that make up the program. Other factors contributing to the conciseness are its functional programming constructs, as well as its macro system. These will be discussed in more detail below.

Functional Programming

Functional programming is a very broad topic, but one of the main concepts is the idea of first-class functions. What this means is that the programming language can define functions that are used as values throughout the program. These functions can interact with other functions by being passed along as arguments or returned as values. This differs from programming in an object-oriented program like Java, where every function (or method) must be contained within an object, which is then passed along as an argument and returned from methods contained in other objects. By allowing functional programming, Clojure lets you define exactly what your program should do without having to wrap everything in unnecessary class declarations.

Immutable, Persistent Data Structures

Clojure forces programs to use immutable data structures by default. While at first this may seem like a limitation of the language, it actually turns out to be an advantage, because it avoids the incidental complexity that is involved with mutable data structures. A data structure created in one part of a program is usually passed along to another part of the program to be used in some other computation. If that data structure can be modified during its lifecycle, it makes the program harder to reason about, and the programmer has to use more mental energy to account for the various transformations that the data might undergo. With immutable data structures, the programmer can take it for granted that the data structure will never change. Immutable data structures are even more important in multi-threaded environments, discussed in the next section.

Concurrency

Advances in technology have brought multi-core processing into all facets of computing, down to the hardware that runs mobile phones. Multiple cores allow programs to create different threads of execution to be performed simultaneously. In many cases, these threads will all want to share the same data structure. In Java there are several tools to synchronize a program across multiple threads, but these tools can be unwieldy and in some cases introduce performance penalties and are hard to diagnose bugs. Clojure provides a number of features to safely and efficiently design multi-threaded programs. One feature was mentioned above: immutable data structures. If data is immutable, there is no need to provide any coordination across threads since the underlying data cannot change. In most cases, however, it is desirable for a program to share some amount of state across its threads of execution. For these instances, Clojure provides a tool called software transactional memory (STM). This is a mechanism similar to how transactions are processed in a database and allows for safe coordination of updates to application data to prevent it from reaching an inconsistent state.

Macros

Macros are a powerful feature of Clojure and other Lisps that allow the creation of code that manipulates other code. Stated slightly differently, a macro is code that takes code as input and performs some transformation which returns an expanded version of the code as output. With macros, a program can create new syntax rules and language constructs to make a specific task easier to implement. Macros can even be used to create wholly new languages, known as Domain Specific Languages, which might cater to a particular class of programming tasks.

Hopefully the information in this article has revealed a number of advantages to using Clojure in your next software project. In some cases, just the ability to concisely construct a program is enough of an advantage to warrant choosing Clojure. With additional requirements for concurrent execution, Clojure becomes an even more compelling choice.