Tuesday, January 16, 2018

Using Google's Protocol Buffers with Java

Effective Java, Third Edition was recently released and I have been interested in identifying the updates to this class Java development book whose last edition only covered through Java 6. There are obviously completely new items in this edition that are closely related to Java 7, Java 8, and Java 9 such as Items 42 through 48 in Chapter 7 ("Lambdas and Streams"), Item 9 ("Prefer try-with-resources to try-finally"), and Item 55 ("Return optionals judiciously"). I was (very slightly) surprised to realize that the third edition of Effective Java had a new item not specifically driven by the new versions of Java, but that was instead was driven by developments in the software development world independent of the versions of Java. That item, Item 85 ("Prefer alternatives to Java Serialization") is what motivated me to write this introductory post on using Google's Protocol Buffers with Java.

In Item 85 of Effective Java, Third Edition, Josh Bloch emphasizes in bold text the following two assertions related to Java serialization:

  1. "The best way to avoid serialization exploits is to never deserialize anything."
  2. "There is no reason to use Java serialization in any new system you write."

After outlining the dangers of Java deserialization and making these bold statements, Bloch recommends that Java developers employ what he calls (to avoid confusion associated with the term "serialization" when discussing Java) "cross-platform structured-data representations." Bloch states that the leading offerings in this category are JSON (JavaScript Object Notation) and Protocol Buffers (protobuf). I found this mention of Protocol Buffers to be interesting because I've been reading about and playing with Protocol Buffers a bit lately. The use of JSON (even with Java) is exhaustively covered online. I feel like awareness of Protocol Buffers may be less among Java developers than awareness of JSON and so feel like a post on using Protocol Buffers with Java is warranted.

Google's Protocol Buffers is described on its project page as "a language-neutral, platform-neutral extensible mechanism for serializing structured data." That page adds, "think XML, but smaller, faster, and simpler." Although one of the advantages of Protocol Buffers is that they support representing data in a way that can be used by multiple programming languages, the focus of this post is exclusively on using Protocol Buffers with Java.

There are several useful online resources related to Protocol Buffers including the main project page, the GitHub protobuf project page, the proto3 Language Guide (proto2 Language Guide is also available), the Protocol Buffer Basics: Java tutorial, the Java Generated Code Guide, the Java API (Javadoc) Documentation, the Protocol Buffers release page, and the Maven Repository page. The examples in this post are based on Protocol Buffers 3.5.1.

The Protocol Buffer Basics: Java tutorial outlines the process for using Protocol Buffers with Java. It covers a lot more possibilities and things to consider when using Java than I will cover here. The first step is to define the language-independent Protocol Buffers format. This a done in a text file with the .proto extension. For my example, I've described my protocol format in the file album.proto which is shown in the next code listing.

album.proto

syntax = "proto3";

option java_outer_classname = "AlbumProtos";
option java_package = "dustin.examples.protobuf";

message Album
{
  string title = 1;
  repeated string artist = 2;
  int32 release_year = 3;
  repeated string song_title = 4;
}

Although the above definition of a protocol format is simple, there's a lot covered. The first line explicitly states that I'm using proto3 instead of the assumed default proto2 that is currently used when this is not explicitly specified. The two lines beginning with option are only of interest when using this protocol format to generate Java code and they indicate the name of the outermost class and the package of that outermost class that will be generated for use by Java applications to work with this protocol format.

The "message" keyword indicates that this structure, named "Album" here, is what needs to be represented. There are four fields in this construct with three of them being string format and one being an integer (int32). Two of the four fields can exist more than once in a given message because they are annotated with the repeated reserved word. Note that I created this definition without considering Java except for the two options that specify details of generation of Java classes from this format specification.

The album.proto file shown above now needs to be "compiled" into the Java source class file (AlbumProtos.java in the dustin.examples.protobuf package) that will allow for writing and reading Protocol Buffers's binary format that corresponds to the defined protocol format. This generation of Java source code file is accomplished using the protoc compiler that is included in the appropriate operating system-based archive file. In my case, because I'm running this example in Windows 10, I downloaded and unzipped protoc-3.5.1-win32.zip to get access to this protoc tool. The next image depicts my running protoc against album.proto with the command protoc --proto_path=src --java_out=dist\generated album.proto.

For running the above, I had my album.proto file in the src directory pointed to by --proto_path and I had a created (but empty) directory called build\generated for the generated Java source code to be placed in as specified by --java_out flag.

The generated class's Java source code file AlbumProtos.java in the specified package has more than 1000 lines and I won't list that generated class source code here, but it's available on GitHub. Among the several interesting things to note about this generated code is the lack of import statements (fully qualified package names used instead for all class references). More details regarding the Java source code generated by protoc is available in the Java Generated Code guide. It's important to note that this generated class AlbumProtos has still not been influenced by any of my own Java application code and is solely generated from the album.proto text file shown earlier in the post.

With the generated Java source code available for AlbumProtos, I now add the directory in which this class was generated to my IDE's source path because I'm treating it as a source code file now. I could have alternatively compiled it into a .class or .jar to use as a library. With this generated Java source code file now in my source path, I can build it alongside my own code.

Before going further in this example, we need a simple Java class to represent with Protocol Buffers. For this, I'll use the class Album that is defined in the next code listing (also available on GitHub).

Album.java

package dustin.examples.protobuf;

import java.util.ArrayList;
import java.util.List;

/**
 * Music album.
 */
public class Album
{
   private final String title;

   private final List<String> artists;

   private final int releaseYear;

   private final List<String> songsTitles;

   private Album(final String newTitle, final List<String> newArtists,
                 final int newYear, final List<String> newSongsTitles)
   {
      title = newTitle;
      artists = newArtists;
      releaseYear = newYear;
      songsTitles = newSongsTitles;
   }

   public String getTitle()
   {
      return title;
   }

   public List<String> getArtists()
   {
      return artists;
   }

   public int getReleaseYear()
   {
      return releaseYear;
   }

   public List<String> getSongsTitles()
   {
      return songsTitles;
   }

   @Override
   public String toString()
   {
      return "'" + title + "' (" + releaseYear + ") by " + artists + " features songs " + songsTitles;
   }

   /**
    * Builder class for instantiating an instance of
    * enclosing Album class.
    */
   public static class Builder
   {
      private String title;
      private ArrayList<String> artists = new ArrayList<>();
      private int releaseYear;
      private ArrayList<String> songsTitles = new ArrayList<>();

      public Builder(final String newTitle, final int newReleaseYear)
      {
         title = newTitle;
         releaseYear = newReleaseYear;
      }

      public Builder songTitle(final String newSongTitle)
      {
         songsTitles.add(newSongTitle);
         return this;
      }

      public Builder songsTitles(final List<String> newSongsTitles)
      {
         songsTitles.addAll(newSongsTitles);
         return this;
      }

      public Builder artist(final String newArtist)
      {
         artists.add(newArtist);
         return this;
      }

      public Builder artists(final List<String> newArtists)
      {
         artists.addAll(newArtists);
         return this;
      }

      public Album build()
      {
         return new Album(title, artists, releaseYear, songsTitles);
      }
   }
}

With a Java "data" class defined (Album) and with a Protocol Buffers-generated Java class available for representing this album (AlbumProtos.java), I'm ready to write Java application code to "serialize" the album information without using Java serialization. This application (demonstration) code resides in the AlbumDemo class which is available on GitHub and from which I'll highlight relevant portions of in this post.

We need to generate a sample instance of Album to use in this example and this is accomplished with the next hard-coded listing.

Generating Sample Instance of Album

/**
 * Generates instance of Album to be used in demonstration.
 *
 * @return Instance of Album to be used in demonstration.
 */
public Album generateAlbum()
{
   return new Album.Builder("Songs from the Big Chair", 1985)
      .artist("Tears For Fears")
      .songTitle("Shout")
      .songTitle("The Working Hour")
      .songTitle("Everybody Wants to Rule the World")
      .songTitle("Mothers Talk")
      .songTitle("I Believe")
      .songTitle("Broken")
      .songTitle("Head Over Heels")
      .songTitle("Listen")
      .build();
}

The Protocol Buffers generated class AlbumProtos includes a nested AlbumProtos.Album class that I'll be using to store the contents of my Album instance in binary form. The next code listing demonstrates how this is done.

Instantiating AlbumProtos.Album from Album

final Album album = instance.generateAlbum();
final AlbumProtos.Album albumMessage
   = AlbumProtos.Album.newBuilder()
      .setTitle(album.getTitle())
      .addAllArtist(album.getArtists())
      .setReleaseYear(album.getReleaseYear())
      .addAllSongTitle(album.getSongsTitles())
      .build();

As the previous code listing demonstrates, a "builder" is used to populate the immutable instance of the class generated by Protocol Buffers. With a reference to this instance, I can now easily write the contents of the instance out in Protocol Buffers's binary form using the method toByteArray() on that instance as shown in the next code listing.

Writing Binary Form of AlbumProtos.Album

final byte[] binaryAlbum = albumMessage.toByteArray();

Reading a byte[] array back into an instance of Album can be accomplished as shown in the next code listing.

Instantiating Album from Binary Form of AlbumProtos.Album

/**
 * Generates an instance of Album based on the provided
 * bytes array.
 *
 * @param binaryAlbum Bytes array that should represent an
 *    AlbumProtos.Album based on Google Protocol Buffers
 *    binary format.
 * @return Instance of Album based on the provided binary form
 *    of an Album; may be {@code null} if an error is encountered
 *    while trying to process the provided binary data.
 */
public Album instantiateAlbumFromBinary(final byte[] binaryAlbum)
{
   Album album = null;
   try
   {
      final AlbumProtos.Album copiedAlbumProtos = AlbumProtos.Album.parseFrom(binaryAlbum);
      final List<String> copiedArtists = copiedAlbumProtos.getArtistList();
      final List<String> copiedSongsTitles = copiedAlbumProtos.getSongTitleList();
      album = new Album.Builder(
         copiedAlbumProtos.getTitle(), copiedAlbumProtos.getReleaseYear())
         .artists(copiedArtists)
         .songsTitles(copiedSongsTitles)
         .build();
   }
   catch (InvalidProtocolBufferException ipbe)
   {
      out.println("ERROR: Unable to instantiate AlbumProtos.Album instance from provided binary data - "
         + ipbe);
   }
   return album;
}

As indicated in the last code listing, a checked exception InvalidProtocolBufferException can be thrown during the invocation of the static method parseFrom(byte[]) defined in the generated class. Obtaining a "deserialized" instance of the generated class is essentially a single line and the rest of the lines are getting data out of the instantiation of the generated class and setting that data in the original Album class's instance.

The demonstration class includes two lines that print out the contents of the original Album instance and the instance ultimately retrieved from the binary representation. These two lines include invocations of System.identityHashCode() on the two instances to prove that they are not the same instance even though their contents match. When this code is executed with the hard-coded Album instance details shown earlier, the output looks like this:

BEFORE Album (1323165413): 'Songs from the Big Chair' (1985) by [Tears For Fears] features songs [Shout, The Working Hour, Everybody Wants to Rule the World, Mothers Talk, I Believe, Broken, Head Over Heels, Listen]
 AFTER Album (1880587981): 'Songs from the Big Chair' (1985) by [Tears For Fears] features songs [Shout, The Working Hour, Everybody Wants to Rule the World, Mothers Talk, I Believe, Broken, Head Over Heels, Listen]

From this output, we see that the relevant fields are the same in both instances and that the two instances truly are unique. This is a bit more work than using Java's "nearly automatic" Serialization mechanism implementing the Serializable interface, but there are important advantages associated with this approach that can justify the cost. In Effective Java, Third Edition, Josh Bloch discusses the security vulnerabilities associated with deserialization in Java's default mechanism and asserts that "There is no reason to use Java serialization in any new system you write."

Monday, January 15, 2018

Easy Fine-Grained Sorting with JDK 8

Java 8's introduction of streams and useful static/default methods on the Comparator interface make it easy to compare two objects based on individual fields' values without need to implement a compare(T,T) method on the class whose objects are being compared.

I'm going to use a simple Song class to help demonstrate this and its Song.java code listing is shown next.

Song.java

package dustin.examples.jdk8;

/**
 * Simple class encapsulating details related to a song
 * and intended to be used for demonstration of JDK 8.
 */
public class Song
{
   /** Song title. */
   private final String title;

   /** Album on which song was originally included. */
   private final String album;

   /** Song's artist. */
   private final String artist;

   /** Year song was released. */
   private final int year;

   /**
    * Constructor accepting this instance's title, artist, and release year.
    *
    * @param newTitle Title of song.
    * @param newAlbum Album on which song was originally included.
    * @param newArtist Artist behind this song.
    * @param newYear Year song was released.
    */
   public Song(final String newTitle, final String newAlbum,
               final String newArtist, final int newYear)
   {
      title = newTitle;
      album = newAlbum;
      artist = newArtist;
      year = newYear;
   }

   public String getTitle()
   {
      return title;
   }

   public String getAlbum()
   {
      return album;
   }

   public String getArtist()
   {
      return artist;
   }

   public int getYear()
   {
      return year;
   }

   @Override
   public String toString()
   {
      return "'" + title + "' (" + year + ") from '" + album + "' by " + artist;
   }
}

The Song class whose listing was just shown lacks a compare method, but we can still compare instances of this class in JDK 8 very easily. Based on the class definition of Song just shown, the following code can be used to sort a List of song instances based, in order, on year released, artist, and finally album.

Sorting List of Songs by Year, Artist, and Album (in that order)

/**
 * Returns a sorted version of the provided List of Songs that is
 * sorted first by year of song's release, then sorted by artist,
 * and then sorted by album.
 *
 * @param songsToSort Songs to be sorted.
 * @return Songs sorted, in this order, by year, artist, and album.
 */
private static List<Song> sortedSongsByYearArtistAlbum(
   final List<Song> songsToSort)
{
   return songsToSort.stream()
      .sorted(
         Comparator.comparingInt(Song::getYear)
                   .thenComparing(Song::getArtist)
                   .thenComparing(Song::getAlbum))
      .collect(Collectors.toList());
}

The above code listing would have been slightly less verbose had I statically imported the Comparator and the Collectors, but it's still fairly concise to include those interface and class names in the listing and probably more useful for an introductory blog post on this subject.

In the above code listing, the static default methods Comparator.comparingInt and Comparator.thenComparing are used to sort the stream of Song associated with the underlying List by year, and then by artist, and finally by album. The code is highly readable and allows for comparison of objects (and resulting sorting of those instances) based on arbitrary individual accessor methods without need for an explicitly specified Comparator (natural sorting order used for each compared accessor result). Note that if an explicit Comparator is desired, it can be provided to these static default methods via overloaded methods of the same name that accept a Comparator.

The next code listing is the entire demonstration class. It includes the method just shown and also shows the contrived example constructed of an unsorted List of songs.

FineGrainSortingDemo.java

package dustin.examples.jdk8;

import static java.lang.System.out;

import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
import java.util.stream.Collectors;

/**
 * Demonstration of easy fine-grained sorting in JDK 8 via
 * stream support for sorting and Comparator's static and
 * default method implementations.
 */
public class FineGrainSortingDemo
{
   /**
    * Construct List of {@code Song}s.
    * 
    * @return Instances of {@code Song}.
    */
   private static List<Song> generateSongs()
   {
      final ArrayList<Song> songs = new ArrayList<>();
      songs.add(
         new Song(
            "Photograph",
            "Pyromania",
            "Def Leppard",
            1983));
      songs.add(
         new Song(
            "Hysteria",
            "Hysteria",
            "Def Leppard",
            1987));
      songs.add(
         new Song(
            "Shout",
            "Songs from the Big Chair",
            "Tears for Fears",
            1984));
      songs.add(
         new Song(
            "Everybody Wants to Rule the World",
            "Songs from the Big Chair",
            "Tears for Fears",
            1985));
      songs.add(
         new Song(
            "Head Over Heels",
            "Songs from the Big Chair",
            "Tears for Fears",
            1985
         ));
      songs.add(
         new Song(
            "Enter Sandman",
            "Metallica",
            "Metallica",
            1991
         )
      );
      songs.add(
         new Song(
            "Money for Nothing",
            "Brothers in Arms",
            "Dire Straits",
            1985
         )
      );
      songs.add(
         new Song(
            "Don't You (Forget About Me)",
            "A Brass Band in African Chimes",
            "Simple Minds",
            1985
         )
      );
      return songs;
   }

   /**
    * Returns a sorted version of the provided List of Songs that is
    * sorted first by year of song's release, then sorted by artist,
    * and then sorted by album.
    *
    * @param songsToSort Songs to be sorted.
    * @return Songs sorted, in this order, by year, artist, and album.
    */
   private static List<Song> sortedSongsByYearArtistAlbum(
      final List<Song> songsToSort)
   {
      return songsToSort.stream()
         .sorted(
            Comparator.comparingInt(Song::getYear)
                      .thenComparing(Song::getArtist)
                      .thenComparing(Song::getAlbum))
         .collect(Collectors.toList());
   }

   /**
    * Demonstrate fine-grained sorting in JDK 8.
    *
    * @param arguments Command-line arguments; none expected.
    */
   public static void main(final String[] arguments)
   {
      final List<Song> songs = generateSongs();
      final List<Song> sortedSongs = sortedSongsByYearArtistAlbum(songs);
      out.println("Original Songs:");
      songs.stream().forEach(song -> out.println("\t" + song));
      out.println("Sorted Songs");
      sortedSongs.forEach(song -> out.println("\t" + song));
   }
}

The output from running the above code is shown next and lists the newly ordered Songs after using the sorting code. It's worth noting that this stream.sorted() operation does not change the original List (it acts upon the stream rather than upon the List).

Original Songs:
 'Photograph' (1983) from 'Pyromania' by Def Leppard
 'Hysteria' (1987) from 'Hysteria' by Def Leppard
 'Shout' (1984) from 'Songs from the Big Chair' by Tears for Fears
 'Everybody Wants to Rule the World' (1985) from 'Songs from the Big Chair' by Tears for Fears
 'Head Over Heels' (1985) from 'Songs from the Big Chair' by Tears for Fears
 'Enter Sandman' (1991) from 'Metallica' by Metallica
 'Money for Nothing' (1985) from 'Brothers in Arms' by Dire Straits
 'Don't You (Forget About Me)' (1985) from 'A Brass Band in African Chimes' by Simple Minds
Sorted Songs
 'Photograph' (1983) from 'Pyromania' by Def Leppard
 'Shout' (1984) from 'Songs from the Big Chair' by Tears for Fears
 'Money for Nothing' (1985) from 'Brothers in Arms' by Dire Straits
 'Don't You (Forget About Me)' (1985) from 'A Brass Band in African Chimes' by Simple Minds
 'Everybody Wants to Rule the World' (1985) from 'Songs from the Big Chair' by Tears for Fears
 'Head Over Heels' (1985) from 'Songs from the Big Chair' by Tears for Fears
 'Hysteria' (1987) from 'Hysteria' by Def Leppard
 'Enter Sandman' (1991) from 'Metallica' by Metallica

JDK 8's introduction of streams and default and static methods in interfaces (particularly on Comparator in this case) make it easy to compare two objects field-by-field in a desirable order without any explicit Comparator other than the pre-built static default methods on the Comparator interface if the fields being compared have a desired natural order.

Thursday, January 11, 2018

Converting Collections to Maps with JDK 8

I have run into the situation several times where it is desirable to store multiple objects in a Map instead of a Set or List because there are some advantages from using a Map of unique identifying information to the objects. Java 8 has made this translation easier than ever with streams and the Collectors.toMap(...) methods.

One situation in which it has been useful to use a Map instead of a Set is when working with objects that lack or have sketchy equals(Object) or hashCode() implementations, but do have a field that uniquely identifies the objects. In those cases, if I cannot add or fix the objects' underlying implementations, I can gain better uniqueness guarantees by using a Map of the uniquely identifying field of the class (key) to the class's instantiated object (value). Perhaps a more frequent scenario when I prefer Map to List or Set is when I need to lookup items in the collection by a specific uniquely identifying field. A map lookup on a uniquely identifying key is speedy and often much faster than depending on iteration and comparing each object with invocation to the equals(Object) method.

With JDK 8, it's easier than ever to construct a Map from an existing List or Set. To help demonstrate this, a simple Book class will be used. This Book class does not override equals(Object) or hashCode() from the Object class and so is not an appropriate class to use in a Set or as a Map key. However, its getIsbn() method returns an International Standard Book Number that is assumed unique for purposes of this demonstration.

Book.java

package dustin.examples.jdk8;

/**
 * Represents a book, but does not override {@code equals(Object)}
 * or {@code hashCode()}.
 */
public class Book
{
   /** International Standard Book Number (ISBN-13). */
   final String isbn;

   /** Title of book. */
   final String title;

   /** Edition of book. */
   final int edition;

   /**
    * Constructor.
    *
    * @param newIsbn International Standard Book Number (-13).
    * @param newTitle Title.
    * @param newEdition Edition.
    */
   public Book(final String newIsbn, final String newTitle, final int newEdition)
   {
      isbn = newIsbn;
      title = newTitle;
      edition = newEdition;
   }

   /**
    * Provide ISBN-13 identifier associated with this book.
    *
    * @return ISBN-13 identifier.
    */
   public String getIsbn()
   {
      return isbn;
   }

   /**
    * Provide title of this book.
    *
    * @return Book's title.
    */
   public String getTitle()
   {
      return title;
   }

   /**
    * Provide edition of this book.
    *
    * @return Book's edition.
    */
   public int getEdition()
   {
      return edition;
   }

   @Override
   public String toString()
   {
      return title + " (Edition " + edition + ") - ISBN-13: " + isbn;
   }
}

With this class in place, the demonstration class CollectionToMapDemo shows how easy it is with JDK 8 to convert various Java collection types (Set, List, and even arrays) to a Map.

CollectionToMapDemo.java

package dustin.examples.jdk8;

import static java.lang.System.out;

import java.util.Arrays;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;

/**
 * Demonstrates conversion of Java collections to Java Maps.
 */
public class CollectionToMapDemo
{
   /**
    * Multiple instances of Book, a class that lacks a proper
    * equals(Object) method, but for which its getIsbn() method
    * is assumed to return a unique identifier for each instance.
    */
   private static final Book[] books;

   static
   {
      books = new Book[]
         {
            new Book("978-0-201-31005-4", "Effective Java", 1),
            new Book("978-0-321-35668-0", "Effective Java", 2),
            new Book("978-0-13-468599-1", "Effective Java", 3)
         };
   }

   /**
    * Convert provided array of Book instances to Map of each Book's ISBN to
    * that instance of the Book.
    * 
    * @param booksArray Array of Book instances.
    * @return Map of each book's ISBN (key) to the book's full instance (value).
    */
   private static Map<String, Book> convertArrayToMap(final Book[] booksArray)
   {
      return Arrays.stream(books).collect(
         Collectors.toMap(Book::getIsbn, book -> book));
   }

   /**
    * Convert provided List of Book instances to Map of each Book's ISBN to
    * that instance of the Book.
    *
    * @param booksList List of Book instances.
    * @return Map of each book's ISBN (key) to the book's full instance (value).
    */
   private static Map<String, Book> convertListToMap(final List<Book> booksList)
   {
      return booksList.stream().collect(
         Collectors.toMap(Book::getIsbn, book -> book));
   }

   /**
    * Convert provided Set of Book instances to Map of each Book's ISBN to
    * that instance of the Book.
    *
    * @param booksSet Set of Book instances.
    * @return Map of each book's ISBN (key) to the book's full instance (value).
    */
   private static Map<String, Book> convertSetToMap(final Set<Book> booksSet)
   {
      return booksSet.stream().collect(
         Collectors.toMap(Book::getIsbn, book -> book));
   }

   public static void main(final String[] arguments)
   {
      out.println("ARRAY->MAP:\n" + convertArrayToMap(books));

      final List<Book> booksList = Arrays.asList(books);
      out.println("LIST->MAP:\n" + convertListToMap(booksList));

      final Set<Book> booksSet
         = new HashSet<>(Arrays.stream(books).collect(Collectors.toSet()));
      out.println("SET->MAP:\n" + convertSetToMap(booksSet));
   }
}

The most important methods in the class listing just shown are convertArrayToMap(Book[]), convertListToMap(List<Book>), and convertSetToMap(Set<Book>). All three implementations are the same once a stream based on the underlying Set, List, or array has been accessed. In all three cases, it's merely a matter of using one of the stream's collect() method (a reduction operator that is typically preferable over sequential iteration) and passing it an implementation of the Collector interface that is provided via a predefined toMap() Collector from the Collectors class.

The output from running this demonstration class against the instances of Book is shown next:

ARRAY->MAP:
{978-0-201-31005-4=Effective Java (Edition 1) - ISBN-13: 978-0-201-31005-4, 978-0-321-35668-0=Effective Java (Edition 2) - ISBN-13: 978-0-321-35668-0, 978-0-13-468599-1=Effective Java (Edition 3) - ISBN-13: 978-0-13-468599-1}
LIST->MAP:
{978-0-201-31005-4=Effective Java (Edition 1) - ISBN-13: 978-0-201-31005-4, 978-0-321-35668-0=Effective Java (Edition 2) - ISBN-13: 978-0-321-35668-0, 978-0-13-468599-1=Effective Java (Edition 3) - ISBN-13: 978-0-13-468599-1}
SET->MAP:
{978-0-201-31005-4=Effective Java (Edition 1) - ISBN-13: 978-0-201-31005-4, 978-0-321-35668-0=Effective Java (Edition 2) - ISBN-13: 978-0-321-35668-0, 978-0-13-468599-1=Effective Java (Edition 3) - ISBN-13: 978-0-13-468599-1}

I have run into several situations in which it has been advantageous to have multiple objects in a Map of unique identifier to full instance of those objects, but have been given the objects in a Set, List, or array. Although it's never been particularly difficult to convert these Sets, Lists, and arrays to Maps in Java, it's easier than ever in Java 8 to make this conversion.

Wednesday, January 10, 2018

The Highly Useful Java ChronoUnit Enum

Several years ago, I published the blog post "The Highly Useful Java TimeUnit Enum" that looked at the TimeUnit enum introduced with JDK 5. JDK 8 introduced a newer enum, ChronoUnit, that is better suited than TimeUnit for contexts other than concurrency such as date/time manipulations.

Located in the java.time.temporal package, the ChronoUnit class implements the TemporalUnit interface, an interface used extensively in the highly desired JDK 8-introduced Date/Time API. The blog post "Days Between Dates in Java 8" demonstrates use of this class to calculate periods of time between two instances of Temporal.

The blog post "Java 9. Where 'forever' is hard coded." looks at "two new methods in the TimeUnit class" for JDK 9. These methods, toChronoUnit() and of(ChronoUnit), support translation of TimeUnit to a ChronoUnit and translation of ChronoUnit to TimeUnit. Not all values in ChronoUnit can be translated to an equivalent in TimeUnit, in which cases an IllegalArgumentException is thrown.

The Javadoc comments on each value in ChronoUnit describe what unit of time each value represents. However, it's interesting to me to see what Duration is returned for each value in ChronoUnit. The following code snippet will write these Duration's toString() representations to standard output for all values in the ChronoUnit enum.

Displaying Durations of ChronoUnits

for (final ChronoUnit unit : ChronoUnit.values())
{
   final Duration duration = unit.getDuration();
   out.println(unit + ": " + duration + " (" + duration.getSeconds() + " seconds)");
}

When executed, the above code produces the following output:

Nanos: PT0.000000001S (0 seconds)
Micros: PT0.000001S (0 seconds)
Millis: PT0.001S (0 seconds)
Seconds: PT1S (1 seconds)
Minutes: PT1M (60 seconds)
Hours: PT1H (3600 seconds)
HalfDays: PT12H (43200 seconds)
Days: PT24H (86400 seconds)
Weeks: PT168H (604800 seconds)
Months: PT730H29M6S (2629746 seconds)
Years: PT8765H49M12S (31556952 seconds)
Decades: PT87658H12M (315569520 seconds)
Centuries: PT876582H (3155695200 seconds)
Millennia: PT8765820H (31556952000 seconds)
Eras: PT8765820000000H (31556952000000000 seconds)
Forever: PT2562047788015215H30M7.999999999S (9223372036854775807 seconds)

The "PT" prefix on each of the Duration's string representations shown above indicates that the representation is a "period" duration designation ("P") and a "time" designation ("T") per the ISO-8601 standard. The "S", "M", and "H" are seconds, minutes, and hours respectively. The values of ChronoUnit that represent time units less than a second (NANOS, MICROS, and MILLIS) show "0 seconds" because they are less than 1 second and the returned value is an integral long.

The Javadoc comments on each value defined in the ChronoUnit class are well written. They follow what in my mind is a Javadoc "best practice": place a concise but informative initial sentence in the Javadoc to show up in the "Method Summary" section of the generated HTML page and place additional useful details in sentences after that initial summary sentence. For example, the Javadoc comment for ChronoUnit.ERAS states, "Unit that represents the concept of an era. The ISO calendar system doesn't have eras thus it is impossible to add an era to a date or date-time. The estimated duration of the era is artificially defined as 1,000,000,000 Years. When used with other calendar systems there are no restrictions on the unit." The bolded sentence (I added that emphasis) is what shows up in the "Method Summary" and the entire text shown here is displayed above the method in its complete explanation.

One of the more interesting values in the ChronoUnit enum is FOREVER. As the output of the code listing above demonstrated, the FOREVER value has a Duration of "PT2562047788015215H30M7.999999999S", corresponding to 2562047788015215 hours, 30 minutes, and 7.999999999 seconds. Or, as Grzegorz Gajos expresses it, "Java defines forever as 9 223 372 036 854 775 807 seconds. Which is 2.92277266 × 1011 years. Better be sure to schedule Java upgrade in your application before times run out."

When would ChronoUnit.FOREVER be useful? Its Javadoc-based description explains its primary reason for existence: "Artificial unit that represents the concept of forever. This is primarily used with TemporalField to represent unbounded fields such as the year or era. The estimated duration of the era is artificially defined as the largest duration supported by Duration."

TimeUnit is a useful enum for working with Java's concurrency constructs and could be used in contexts other than concurrency as long as some severe limitations for these other contexts were considered. The JDK 8-introduced ChronoUnit is a better enum for these non-concurrency contexts and is especially designed for use with the JDK 8 Date/Time API.

Sunday, December 31, 2017

Significant Software Development Developments of 2017

This post is my personal and opinionated assessment of some of the most significant developments related to software development in 2017. This is my eleventh year for this annual post and my previous years' assessment are available for 2016, 2015, 2014, 2013, 2012, 2011, 2010, 2009, 2008, and 2007. As with these previous years' assessments, this assessment of 2017's major developments in software development are obviously biased, opinionated, and limited to my perspective.

I think it important to re-emphasize that although this is an opinion-heavy post, the opinions are not on whether a particular language, framework, or tool is "best" or "worst." Rather, the opinions come in when deciding which of these frameworks, languages, or tools had the biggest developments of the year. I could, for example, decide that some language I hated had a big year and talk about its significant accomplishments of the year even if I loathe the language. Last year, at least one reader confused my listing of languages as endorsement of that being a better language, but this post is not that at all. Instead, this is a post of significant developments during 2017 in the software development world that I'm aware of. I obviously cannot be aware of every major software development happening in 2017 and welcome any feedback about major developments of 2017 in the world of software development.

One of the many challenges of writing a post such as this is that many of the subjects overlap significantly and it can be difficult to separate them and distinguish them. For example, the Internet of Things (IoT), Edge Computing, Cloud Computing, Big Data, Machine Learning, Artificial Intelligence (AI), and Software Security are all closely related and overlap to a degree.

10. Kotlin

2017 was a significant year for Kotlin. Kotlin 1.1 was released and moved JavaScript support out of "experimental." The release notes state that this JavaScript support "supports all Kotlin language features, a large part of the standard library, as well as JavaScript interoperability." The Kotlin 1.1 release also introduced (experimental) coroutines.

Kotlin 1.2 was also released in 2017 and built on the JavaScript support added by Kotlin 1.1 to add "the possibility to reuse code between the JVM and JavaScript."

The Technology Preview of Kotlin/Native was announced in 2017. The kotlin-native GitHub page describes Kotlin/Native as "an LLVM backend for the Kotlin compiler, runtime implementation, and native code generation facility using the LLVM toolchain" that is "primarily designed to allow compilation for platforms where virtual machines are not desirable or possible (such as iOS or embedded targets), or where a developer is willing to produce a reasonably-sized self-contained program without the need to ship an additional execution runtime."

Perhaps the biggest news for Kotlin in 2017 was its adoption as an officially supported program language for developing Android applications. The Kotlin Blog post "Kotlin on Android. Now official" opens with, "Today, at the Google I/O keynote, the Android team announced first-class support for Kotlin." The Kotlin on Android FAQ states that "Kotlin is fully supported in Android Studio 3.0 and higher." The 2017 4th Quarter Realm Report predicts that "2018 will be the year of Kotlin" and and that Kotlin will overtake Java as primary Android development language by December 2018.

Spring Framework 5 support for Kotlin was also announced in 2017. Wired features an article this year titled "Kotlin: the Upstart Coding Language Conquering Silicon Valley" and the Heroku Blog features a post "On the Rise of Kotlin". There are now multiple books available on Kotlin, most with publication dates in 2017. The Tiobe Index's December 2017 edition states, "The programming languages Kotlin and C seem to be the only candidates to become programming language of the year 2017" (Tiobe's programming language of the year is the "programming language that has the highest rise in ratings in a year"). Gabriela Motroc has published five proofs of "why Kotlin deserves to become the programming language of the year."

9. React

The main page for React.js describes it as "A JavaScript library for building user interfaces." This succinct description is perhaps a bit understated. React has received plenty of praise online this year and his been proclaimed by several blog post and article authors as the most popular choice for building front ends. The post "React.JS Top 10 Articles of The Year (v.2017)" highlights ten posts on React selected from nearly 12 thousand posts published in 2017. Similarly, the post "The most popular React links of 2017" highlights "the most popular links shared in 2017" from the year's editions of the React Status newsletter.

The popularity of React is proven by the influence the library has on general JavaScript development. For example, React's JSX that appears to mix JavaScript and HTML (it's really JavaScript with a HTML-like syntactic facade or "syntactic sugar") appears to be changing some opinions regarding the seeming mixing of JavaScript and HTML and there is now discussion about the differences between "separation of concerns" and "separation of technologies." In "JSX Looks Like An Abomination, But it's Good for You," Eric Elliott writes, "JSX is like a healthy vegetable that tastes like decadent chocolate cake. You feel guilty, but it's good for you."

8. Low-Code/No-Code

I've been a professional software developer long enough now to have seen many trends come and go, come again and go again. Throughout this time, there has been a seemingly constant effort to reduce the amount of code that needs to be written and maintained. We have come along way in many respects, but there is still a lot of custom code that is hand-written and much of it is redundant. Recently (and especially in 2017), the terms "low-code", "no-code", and "citizen developer" have become popular in this constant search for software development's holy grail.

Edward Hadley's "Low-Code Development Platforms Address Soaring Application Needs" explains the renewed recent interest in low-code and no-code solutions. He writes, "While the demand for custom applications has never been higher, traditional development approaches simply can’t keep pace. According to a statistic cited by Gartner, through 2021, market demand for app development will grow at least five times faster than IT capacity to deliver it." He adds, "The inherent value of a low-code development platform is that it brings IT and the business together." Jonathan Hult maintains a collection of "Low-code app builders" that "serve as a roundup post linking to low-code (or no-code) app (web/desktop) builders" and "should be citizen developer friendly."

It is postulated in The Atlantic article "The Coming Software Apocalypse" looks at the advantages of no-code/low-code/citizen developers from a different perspective. A central premise of this article is that many of the current and impending problems associated with software are due to general programmers not understanding well enough the problems they are trying to solve with code. Developers must focus on the instructions to the machine via code more than on the problem being solved. Nancy Leveson is quoted as saying, "The problem is that software engineers don't understand the problem they're trying to solve, and don't care to." She adds that the real problem is not being able to cover all the necessary requirements to be coded to in a manageable way. Bret Victor is quoted as saying, "I'm not sure that programming has to exist at all. Or at least software developers." He maintains that tools should be created to remove the need to work directly with code to solve the problems.

A paper by the Oak Ridge National Laboratory has a title starting with "Will humans even write code in 2040...?" starts its opening "Abstract" section with this statement, "Programming trends suggest that software development will undergo a radical change in the future: the combination of machine learning, artificial intelligence, natural language processing, and code generation technologies will improve in such a way that machines, instead of humans, will write most of their own code by 2040." One of the paper's authors (Jay Jay Billings) states in "Good code generators will be the most helpful and useful tools for coding by 2040" that he believes that "humans will still write code in 2040," but that "most code that we write now - what we might consider 'everyday code' - will be written with code generators that are fed by machine learning, AI, and natural language processing."

7. Blockchain

Primarily because of the mainstream interest in Bitcoin, many more people have become aware of the blockchain concept in 2017.

There are high expectations for blockchain. In "Blockchain: An Introduction," Arun Pandey writes that blockchain "helps transactions to have public witnesses and hence minimize cyber crime and fraud" and "serves as an open, distributed ledger to record transactions between two parties in a verifiable and lasting way." The article "What is Blockchain Technology? A Step-by-Step Guide For Beginners" states that "blockchain is an undeniably ingenious invention" that has "created the backbone of a new type of internet ... by allowing digital information to be distributed but not copied." Gabriela Motroc has asked "eight blockchain influencers" if "blockchain [can] transform the world?"

6. Machine Learning (ML) / Artificial Intelligence (AI) / Big Data

The authors of the Harvard Business Review July 2017 cover story "The Business of Artificial Intelligence" write, "The most important general-purpose technology of our era is artificial intelligence, particularly machine learning (ML) - that is, the machine's ability to keep improving its performance without humans having to explain exactly how to accomplish all the tasks it's given." They cover ways that AI has already helped business, but also look at unrealistic expectations for AI.

Some interesting uses of machine learning are covered in online resources such as Machine Learning for Humans, How artificial intelligence and machine learning will disrupt legal space, Machine learning will not replace people in all jobs: Study, Reimagine Business for Machine Learning, and Falling into Machine Learning.

In "2017: The Year AI Floated into the Cloud," Jackie Snow writes that in 2017 "tech firms opened a new front in the battle to win users over in the cloud: the large-scale introduction of cloud-based AI." Computer Weekly has published its "Top 10 Artificial Intelligence Stories of 2017" and Zach Emmanuel writes, "Artificial intelligence (AI) has continued to gain prominence in 2017 as one of the biggest upcoming technologies."

Big Data remains "big" in terms of popularity. Alex Woodie's post "10 Key Big Data Trends That Drove 2017" details "10 of the biggest takeaways for the big data year that was 2017." The post "Big Data: Main Developments in 2017 and Key Trends in 2018" "considers what happened in Big Data" in 2017. A DZone post details select IT Executives' responses related to their "biggest surprises about big data and analytics" in 2017. The post "Top 10 Big Data Blogs of 2017" looks at SyncSort's top ten blog posts of Big Data in 2017, including the post "Just How Big is Big Data, Anyway?" The article "The rise of big data in 2017: Here are the top emerging trends" begins with, "We saw the rise of big data in 2017 and the trend will continue to gain speed. Accessing and preserving big data is now a common practice at almost all organizations."

One of the "big" stories in the world of Big Data is the European Union's General Data Protection Regulation (GDPR). According to its web page, the GDPR "was designed to harmonize data privacy laws across Europe, to protect and empower all EU citizens data privacy and to reshape the way organizations across the region approach data privacy." That page states that its effective as of 25 May 2018 and "at which time those organizations in non-compliance will face heavy fines." The "key changes" of the GDPR can be found in "GDPR Key Changes." Bozhidar Bozhanov has posted "GDPR - A Practical Guide for Developers."

I might have "cheated" a bit to include artificial intelligence, machine learning, and Big Data all in one item, but they are closely related in many scenarios. Posts and articles talking about these together include "How Big Data Is Empowering AI and Machine Learning?", "Why AI, Machine Learning And Big Data Really Matter To B2B Companies," "Artificial intelligence, machine learning, deep learning and beyond," and "How Big Data Is Empowering AI and Machine Learning at Scale."

5. Containers

Containers continued to be a popular topic in 2017.

The Docker Blog has collected its Top Five Blogs of 2017 that include the posts Build and Run Your First Docker Windows Server Container and Exciting new things for Docker with Windows Server 1709. That blog also features a post Top 5 Docker Customer Success Stories in 2017.

2017 was an especially big year for Kubernetes. Christopher Tozzi writes in "Kubernetes’ Big Triumph of 2017: Killing Docker" that "the biggest change of 2017 in the world of containers" is "Kubernetes’s rise to dominance." He adds, "Kubernetes has taken the container ecosystem by storm. It has basically become the new Docker, in the sense that it is the name that now dominates most conversations about container technology." The Docker page "Adding Kubernetes support in the Docker platform" discusses Kubernetes integration with Docker in greater detail.

Chris Short has written a post Docker, Inc is Dead in which he opens with the sentence, "To say that Docker had a very rough 2017 is an understatement." He adds, "People will look back on 2017 as the year Docker, a great piece of software, was completely ruined by bad business practices leading to its end in 2018."

4. DevOps

The term "DevOps" seems to be written about and talked about more than ever. Everything I wrote about DevOps in last year's post remains true but to an even (much) larger degree than last year: "Although the DevOps concept has been around for a while now, it's gained popularity with the 'suits' (management) and the term is thrown around in all types of contexts. Software product vendors emphasize how their products continue to DevOps and software development requisitions and job listings are filled with references to experience with DevOps."

In my perusal of blog post titles and headlines each day, I see numerous references to DevOps. In just this week, posts on DevOps that have crossed my screen include DevOps remains a competitive advantage, Kubernetes news, DevOps developments dominate 2017 for IT shops, and Top 10 DevOps articles of 2017: The Enterprisers Project. Two (of many) examples of DevOps being embraced by the highest levels of management is the existence of the Forbes article "10 Top DevOps Barriers And Trends Forecasted For 2018," "2017 Brought More Open Standards for DevOps," and the DZone questions and answers format "Concerns With DevOps."

DZone has published "2017 DevOps Surprises" that contains observations from "IT executives" regarding the "biggest surprises about DevOps" in 2017. Forrester called 2017 the year of DevOps and predicts 2018 to be "The Year of Enterprise DevOps." Stanislav Ivaschenko has posted "Hopes and Fails in DevOps 2017."

3. Microservices

I cannot think of a software development related term that I heard or read more about during 2017 than "microservices." The term has been used for several years now, but 2017 seemed to see more use of it than ever. The Register recently featured an article called "Microservices 101" that introduces microservices for those who may not have heard much about them yet. Vendors are pushing microservices, which always means hearing a lot about them. "The State of Microservices Survey 2017 - Eight trends you need to know" provides observations from a survey of developers using microservices.

2. Cloud Computing / Internet of Things (IoT)

Cloud computing and the Internet of Things (IoT) continue to become more pervasive in our lives as consumers and therefore, not surprisingly, as software developers.

Recent articles on cloud computing in 2017 include "The Cloud in 2017: Amazon Web Services shows no signs of slowing during the year of Kubernetes," "2017 Cloud Computing and Data Center Industry Review and Conclusion," "8 resources for understanding the open source cloud in 2017" ("2017 was a big year for open source cloud computing tools"), "Cloud computing: Getting bigger but more complicated too," "Top cloud providers dominate headlines in 2017," "2017 cloud computing headlines show upside, hurdles for CIOs," "The top 10 Thoughts on Cloud articles of 2017," "Cloud Computing - What’s The Big Deal," "10 Benefits and Advantages of Cloud Computing," and forward-looking "5 cloud computing trends for 2018".

Sierra Wireless has published a post called "Internet of Things Year in Review: 2017." Other recent articles on the Internet of Things include "The internet of things: Why it matters," "Rise of the machines: who is the ‘internet of things’ good for?," and "2017 Internet Of Things (IoT) Intelligence Update,"

There have also been recent articles with ominous tones regarding internet of things such as "The Internet of Things' Dangerous Future," "The Internet of Things Needs a Code of Ethics," "Is This a Setback for the Internet of Things?," and "The dark side of the internet of things". Securing devices connected to each other via the internet of things is going to remain (or become in some cases) a high priority.

1. Software Security and Software Outages

I'd love to not have software security issues and software outages not top my list, but 2017 seemed to see even more negative incidences related to software security and software availability than ever. Michael Novinson writes that "data breach activity continued to skyrocket in the first 11 months of 2017, with the numbers of breaches jumping to 1,202, according to a report from the Identity Theft Resource Center and CyberScout," which is "up 10 percent from the 1,093 breaches recording during the entirety of 2016." John Zorabedian writes that 2017 "featured daily news about cyberattacks, data breaches, and software vulnerabilities." He adds, "If it feels like our cybersecurity challenges grow bigger and more complex, year after year, it's more than just a perception."

Here is a list of headlines from 2017 that highlight this increased frequency.

DZone has featured two posts on "2017 Security Surprises": Part 1 and Part 2.

Honorable Mention

The following did not make my top ten, but nevertheless saw major developments in 2017. Many of these items would likely make a different developer's top ten. There is no significance implied by the order of these listed entries in the "Honorable Mention" section.

"Serverless" / Function as a Service (FaaS)

I first started paying attention to the term "serverless" when the Fn Project was announced at JavaOne 2017 and especially after reading 8 Reasons why we built the Fn Project. The Martin Fowler article "Serverless Architectures" introduces these concepts well and the InfoQ article "FaaS, PaaS, and the Benefits of the Serverless Architecture" looks at the benefits of using these and talks about "nanoservices." Matt Watson's "What Is Function-as-a-Service? Serverless Architectures Are Here!" is also a good introduction to this topic and when FaaS and "serverless" might make the most sense to use.

Edge Computing

Often closely associated with Internet of Things (IoT), edge computing is described in GE Digital's What is Edge Computing? blog article as "computing infrastructure that exists close to the sources of data ... [on] devices [that] typically reside away from the centralize computing available in the cloud." The Wikipedia entry on edge computing adds, "Edge computing pushes applications, data and computing power (services) away from centralized points to the logical extremes of a network." The previously cited GE Digital article also differentiates Edge Computing versus Fog Computing and Edge Computing versus Cloud Computing.

Quantum Computing and Microsoft Q#

The concept of quantum computing is nothing new, but Microsoft's series of announcements of its Microsoft Quantum Development Kit Preview in the fourth quarter of 2017 seem to me to be a very significant development for quantum computing in 2017 because it feels like an early step toward taking quantum computing mainstream. The Microsoft Quantum Development Kit includes Microsoft's Q# programming language (including standard library and compiler) along with a "local quantum machine simulator" and a Q#-oriented extension for Visual Studio.

A Nature.com post in early 2017 titled "Quantum computers ready to leap out of the lab in 2017" states, "Quantum computing has long seemed like one of those technologies that are 20 years away, and always will be. But 2017 could be the year that the field sheds its research-only image."

A nice introduction to quantum computing (at least as far as the little I understand about the subject allows me to detect) can be found in the aptly named "An Introduction To Quantum Computing." Microsoft has also provided an introduction in "What is quantum computing?" Forbes has published "15 Things Everyone Should Know About Quantum Computing."

An editorial in The Guardian calls the "the great power contest to develop a quantum computer" the "space race of our times." The following are some events and announcements related to quantum computing that occurred in 2017:

WebAssembly

In "Mozilla Made the Web Better for Developers in 2017," Dustin Driver writes that "by the end of 2017, all the major browsers shipped support for WebAssembly, making the web a speedier and more robust place for everyone." The article "WebAssembly Will Finally Let You Run High-Performance Applications in Your Browser" discusses the history behind WebAssembly and "Introduction to WebAssembly: why should we care?" provides a nice introduction to WebAssembly, what it is, and why it will change web development and the user experience on the web.

Reactive System / Reactive Programming

The Reactive Manifesto was published in September 2014 and the reactive design concept seems to have grown in popularity each year since, especially becoming well-known in 2016 and 2017. The Red Hat Developers Blog features a post "5 Things to Know About Reactive Programming" in which Clement Escoffier defines "reactive programming" as "a development model structured around asynchronous data streams." Escoffier emphasizes that reactive programming is not the same thing as a reactive system: "Using reactive programming does not transform your system into a Reactive System. Reactive Systems are the next level."

TypeScript

TypeScript continued to see increased adoption in 2017. Mary Branscombe's post "Why TypeScript Is Growing More Popular" explains some of the reasons for this rise in popularity. Microsoft Technical Fellow and TypeScript developer Anders Hejlsberg is quoted in this article, "There's no doubt the partnership that we have with the Angular team has helped drive the numbers. ... Lots of other frameworks are using TypeScript at this point. Aurelia, Ionic, NativeScript are all, in one way or another, involved in TypeScript. The Ember framework, the Glimmer framework that was just released is written in TypeScript."

Two other posts in 2017 commenting on the rise of TypeScript popularity are "JavaScript for squares: The incredible rise of TypeScript" and "TypeScript - a trend that keeps on trending."

TypeScript turned five in 2017 and saw several new releases in 2017 as part of its new release cadence:

Vue.js

Vue.js ("The Progressive JavaScript Framework") seems to be a JavaScript framework whose popularity is rising quickly. Kevin Peters has written an extensive post called "Vue.js review of 2017" that leaves little need for me to write more regarding Vue.js in 2017. However, an additional potentially interesting read is "This Week in Numbers: React Leads the Pack, Though Vue.js Makes Big Strides in China."

Angular

Angular 5 (pentagonal-donut) was released in 2017 and "focuses on making Angular smaller, faster, and easier to use," but not everyone is pleased. Angular 5.1 has also been released in 2017 as has beta 5.2 versions.

JAX's "Year in review: Angular in 2017" provides a more detailed month-by-month review of Angular-related news in 2017.

JavaScript

JavaScript continued to be a highly adopted and used programming language with just one evidence of that being the numerous libraries and frameworks associated with JavaScript covered in this very blog post. I won't cover individual developments in the world of JavaScript here because there are so many online resources covering similar material. One such post is "JavaScript in 2017: Year in Review, Predictions for 2018" in which Boris Cherny writes, "2017 has been a wild year for JavaScript and frontend development." The article "The most popular JavaScript links of 2017" highlights top stories in 2017 from the JavaScript Weekly newsletters. "JavaScript: Top Articles in 2017" highlights "popular JavaScript Articles and Tutorials on Codeburst in 2017."

There are numerous "state of JavaScript in 2017" type posts (several of these are interpretations or summaries of the first linked post) such as "The State of JavaScript - 2017" (see also "A Look Back at the State of JavaScript in 2017" and "I just asked 23,000 developers what they think of JavaScript. Here's what I learned."), "JavaScript developers prefer React, want to ditch Angular & are attracted to Vue.js", "State of JavaScript: TypeScript rises, Angular falls," "Brief Analysis of the State of JavaScript 2017 Results," "Angular, React and VueJS - The Rise of Client-Side Frameworks in 2017," and "Developer Ecosystem Survey 2017 - JavaScript."

Node.js

Node.js remains popular in 2017 as proven by posts such as "Node.js State of the Union 2017," "This is what Node.js is used for in 2017 - Survey Results," and "The Node.js Community was amazing in 2017!"

Node.js releases in 2017 range from Node.js 7.4.0 through Node.js 9.3.0.

Java

Mark Little of Red Hat is quoted in "Java: 2017 Surprises and 2018 Predictions", "This has been a big year for Java, with several significant developments contributing to the ongoing evolution of the technology. The most notable event in 2017 was Oracle’s announcement to more fully open up Java EE by moving it to an open source foundation - and the subsequent announcement that it had selected the Eclipse Foundation to host the initiative as a top-level project called Eclipse Enterprise for Java (EE4J)."

JDK 9 was released in 2017 with its much anticipated and long-awaited introduction of built-in platform modularity. There was a surprising amount of drama surrounding the naming of future versions of Java SE as well, but it seems that "JDK 10" has won out. Oracle has also announced changes the releases of versions of Java and differentiated between the "Oracle JDK ("commercial long term support offering" intended for "commercial and support customers") and OpenJDK (which will include access to "previously commercial features such as Java Flight Recorder").

IBM open sourcing its JVM in 2017 and it is called OpenJ9. Another big deal to the Java community in 2017 was the publication of the Third Edition of Effective Java, the first edition of this book covering very significant Java versions 7, 8, and 9. Java Code Geeks has posted "Top 10 JavaCodeGeeks posts for 2017" that presents "a compilation with the most popular posts for this year" including "JDK 9 is Feature Complete" and "JDK 9 is the End of the Road for Some Features."

See JAX's Year in Review: Java in 2017 for a detailed analysis of events in the world of Java that occurred in 2017. Ben Evans's article "Looking Forward to Java in 2018" looks briefly at 2017 for Java and then looks at what the future may hold for Java. JAX also has posted "Top 10 Java stories of 2017: Angular, Eclipse, ML, and more."

Swift

Swift 4 was released in 2017. Two resources for details on Swift 4 features are What's New in Swift 4? and What's new in Swift 4.0.

Guy Daher, a self-described "Swift developer evangelist," looks at What Stats and Surveys are saying about Swift in 2017.

Python

Python's popularity continues to increase and it's one of the most-used programming languages available. The State of the Octoverse 2017 states that "Python replaced Java as the second-most popular language on GitHub, with 40 percent more pull requests opened this year than last." Python remains the fourth-most searched for programming language on the Tiobe Index. Python also topped IEEE Spectrum's The 2017 Top Programming Languages (July 2017). As evidence of Python's growing familiarity, there is word that "Microsoft Considers Adding Python as an Official Scripting Language to Excel."

Versions of Python released in 2017 include Python 3.3.7 ("final release" of Python 3.3.x that "has reached end-of-life"), Python 3.4.7, Python 3.5.3, Python 3.6.3, and Python 3.6.4.

Scala

Scala 2.12.4 was released and includes "improved Java 9 friendliness."

The first release of Scala Native (0.1) was announced in 2017 and its GitHub page states, "Your favorite language gets closer to bare metal." The main Scala Native page describes Scala Native as "an optimizing ahead-of-time compiler and lightweight managed runtime designed specifically for Scala."

Ammonite, an open source project dedicated to scripting with Scala, moved toward its 1.0 release in 2017 and Ammonite 1.0.3 is available at time of this writing.

Rust

The Rust programming language continues to grow in popularity in 2017. The Rust Programming Language Blog features a post "Rust in 2017: what we achieved" that covers the highlights of achieving Rust goals in 2017 related to a "single overarching theme" of "increasing productivity, especially for newcomers to Rust." Jimmy Cuadra posted in 2017 "The highs and lows of Rust (2017)." The blog posts "Entering the Quantum Era - How Firefox got fast again and where it’s going to get faster" and "Mozilla Made the Web Better for Developers in 2017" discuss how Rust was an integral part of making the Firefox web browser faster.

Go

The "Go" programming language continues to gain in popularity in the development community. In early 2017, Mariano Gappa wrote about Movio Cinema's Red Squad's adoption of Go. Around the same time, Keval Patel wrote "Why should you learn Go?" IEEE Spectrum lists Go in its "Top Ten Languages for the typical Spectrum reader" and InfoWorld's Paul Krill wrote "Go language soars to new heights in popularity."

The Go programming language in 2017 continued to head towards version 2.0 with releases that included go1.8, go1.9, go1.9.2, and go1.10beta versions.

The open source Joy project (Joy Compiler) for "translat[ing] idiomatic Go into concise Javascript that works in every browser" was introduced in 2017. The Joy FAQ states that it's possible that the Joy Compiler could one day allow for translating Go code to WebAssembly, but there are currently some hurdles to that happening.

In 2017, JetBrains announced the name for its Go IDE: GoLand. GoLand is a commercial (not open source) IDE that "extends the IntelliJ platform with the coding assistance and tool integrations specific for the Go language."

The Golang Weekly Team summarizes Go achievements in 2017 at the beginning of their Golang Weekly Issue 192.

C++

C++17 (also known as C++1z) became "feature complete" in 2017. Bartlomiej Filipek has provided an overview of C++17 features in the appropriately named post "C++ 17 Features" and has posted "7 Features of C++17 that Will Simplify Your Code." The document "Changes between C++14 and C++17 DIS" covers "all the major changes that have been applied to the C++ working draft since the publication of C++14, up to the publication of the C++17..."

The JetBrains Developer Ecosystem Survey 2017 for C++ found that nearly two-thirds of the respondents currently use C++11 and a little more than one-third of the respondents currently use C++14 (respondents must have been allowed to indicate current use of more than one version of C++).

C

C has been named the Tiobe Index's "programming language of the year" for 2017. The "TIOBE Index for January 2018" states that C won this distinction primarily "because there were no outstanding alternatives" and explained at least part of C's resurgence on that index: "A possible reason for this revival is that C is very popular in the growing manufacturing and machine industry (including the automotive market)."

Spring

Josh Long, the Spring Developer Advocate at Pivotal, has written the post "This Year in Spring - 2017" in which he reviews "this very exciting year in Spring and its ecosystem." Some of the Spring-related highlights of 2017 covered in that post include the release of Spring Framework 5.0, release of Pivotal Cloud Foundry (PCF) 2.0, introduction of Spring Cloud Function, and Project riff ("a function service designed to run on Kubernetes").

Spring Framework 5 requires Java 8, includes support for Kotlin, and introduces a reactive programming model based on the Reactor project implementation of the Reactive Streams Specification for the JVM.

JUnit

JUnit 5 was released in 2017 (General Availability in September and 5.1.0-M1 in November). JUnit 5 breaks JUnit tradition and is composed of three sub-projects (JUnit Platform, JUnit Jupiter, and JUnit Vintage). JUnit 5 requires Java 8 to run, but can be used to test Java code built against an older version of Java. Posts written in 2017 regarding JUnit 5 include Embrace JUnit 5, The Basics of JUnit 5 - A Preview (and A Look at JUnit 5's Core Features & New Testing Functionality), The five coolest new features of JUnit 5, What's New In JUnit 5?, and Seven Reasons Why You Should Start Using JUnit 5 Today.

I don't know enough about the software testing world to talk about general software testing developments of 2017, but Cynthia Dunlop's post "The Top Software Testing News of 2017" seems like a good place to start for such an overview. Dunlop opens the article, "It's not every year that 'software testing' hits the headlines across publications such as Forbes, Fortune, and Reuters. But these are no ordinary times for software testing. Agile and DevOps, which are designed to disrupt development, have also thrown software testing into turmoil—causing a shake-up in the software testing tools industry. This is evident by the string of software testing tool company acquisitions, spin-merges, and funding rounds that ended up dominating software testing news throughout 2017." One of the stories referenced and briefly described in that article is the InfoQ article "JUnit 5 is Released."

Munich Switches from Linux Back to Windows

In 2003, the city of Munich decided to move from Windows to Linux. This was an exciting time for Linux advocates in particular, but also for open source advocates in general. Unfortunately, there started to be some bad news coming out of this transition as early as 2004 and it was announced in 2017 that Munich would be returning to Windows. The reactions to this differ depending on who you ask (see "Munich Is Ditching Linux For Purely Political Reasons" and "Munich ends its long-running love affair with Linux"), but there's no doubt this move is a disappointment to the many open source and Linux users who thought the Munich "experiment" would lead the way for other world cities.

The switch from Linux back to Windows is likely far less devastating to Linux and open source adoption in 2017 than it would have been in te 2003-2004 timeframe. Open source and Linux are both well entrenched now and I don't see them losing ground in the near future. However, this is still a big enough story to make my "honorable mention" section and is the #2 story highlighted in the post "Biggest Linux News Stories of 2017."

Ruby

Ruby 2.5 was released on Christmas Day in 2017 (a new release of Ruby on Christmas Day has become an annual tradition). Junichi Ito's post "10 new features in Ruby 2.5" introduces his "handpicked 10 new features" from Ruby 2.5's "new features and performance improvements."

There has been some discussion online about Ruby declining in popularity (Java and other languages have experienced such talk for years). In "Ruby on Rails is out: major coding bootcamp ditches it, due to waning interest," Matthew Hughes writes, "The pivot away from Rails shouldn't surprise anyone. While the framework continues to have a strong, cohesive community, and is actively developed and supported, developer interest has largely shifted elsewhere." Hughes also quotes Zed Shaw and Paul Watson on potential explanations for (slight so far) declining Rails popularity (newer competitors in web development with focus on front-end and single page applications).

PostgreSQL

PostgreSQL 10 was released in late 2017 and includes several new features such as logical replication, declarative table partitioning, improved query parallelism and other general performance improvements, and improved monitoring and control. PostgreSQL versioning is also changing with PostgreSQL 10 to a 2-digit version number instead of the old 3-digit version numbers.

PostgreSQL was named "DBMS of the Year 2017" by DB-Engines. The DB-Engines Ranking is updated monthly and "ranks database management systems according to their popularity." A particular database management system wins the annual prize for having the most significant increase in ranking over the 12 month period.

Conclusion

This post looked at some of the developments in software development that occurred in 2017. I undoubtedly missed some things and missed some developments associated even with some of the listed items. This post will be updated in the coming days and weeks as I remember some of these things or as feedback points out some of these missing items.