Improve your Feedbackloop with Continuous Testing

Have you ever though about what the most valueable thing in software development was for you? And im not talking about things that value for you personally, but for the success of the development itself. Well i have thought about it, and for me it was Feedback - in any form. It is so important, because it enables steering. Development practices are made for the purpose of better feedback. TDD, Continuous Integration, Iterations, to name only a view. Many of the agile methods and XP are basically about better feedback.

It begins with customer interaction. I need as much feedback as possible, as frequent as possible. If i don't get feedback, i'm likely to get on the wrong track, resulting in a product that is not going to be used, because it's not what the customer needed. The more feedback i get, the better the outcome will be. If i get feedback rarely, i am unable to steer. I'm forced to make assumptions which are likely just obstacles. The quicker i get feedback, the faster i can deliver value to my customer. Feedback allows me to steer.

Feedback is as important for programming. I want it early and often. If i write hundreds of lines of code without running them, they will most likely result in a very long and painful debugging session, and a lot of changes. I don't want that, so i take baby steps. They make me go safer, faster and happier. There are two phases that define my programming feedback loop.
  1. The phase where i write code. Lets call it alpha, like so 'α'.
  2. The phase where i evaluate my code, and eventually fix errors. Lets call it beta, like so 'β'.
You could also see those phases as modes. It is important to understand here, that these phases have nothing todo with the alpha/beta definition of a software cycle. I just invented them to describe my programming feedbackloop.In the following graphics you'll notice that the lines get shorter and shorter by example which is intentional and should point out how i got faster using new strategies.
    When i first started coding, i did not write any tests. I wrote lots of code before i tried it out manually. Obviously it didn't work when i first ran it. I ended up in rather long α-phases, where i just wrote code, and also long β-phases, where i evaluated (got my feedback), and fixed it. Like this:



    I was very slow back then.

    I soon started with an interpreted language, which was very cool because i could run the scripts immediately. No need to compile or anything. Just write and run. It shortened my feedback loop, and i became faster overall:



    Sooner or later i eventually started tdd. And regardless of the language that i was using, interpreted or not, it again shortened my feedback loop and made me go faster. The loop was shortened to a single 'unit', which is obviously smaller than anything manually executable. It allowed me to evaluate small behaviours, long before the program was even runnable. It is important to understand, that the α-phase in the following graphic contains both writing tests and implementation. The β-phase is much shorter, since unittests run very fast.


    I thought this was just great, and it could not get any better. Wrong i was!! Later, i tried something that made me go like this:



    "What the ...?" You might ask. No, i did not break space-time. The thing i tried was Continuous Testing. Which basically means, that i do tdd, but i don't run my tests by pressing a button and then wait. I just have them run all the time in the background automatically... Everytime i change my code, my tests immediately run automatically, and show me "OK" or "NOT OK" on a small icon on my screen. Since the tests only take about a second to run, this feedback is instant. And since my IDE saves my files onchange automatically, i do not have to press ctrl+s or anything. I just code... and as i code my files get saved.... and as the files get saved my tests get run... fluently, immediately. This is HUGE. I am now progressing without disruption. I completely broke out of the phases, or if you want to call them 'modes'. I love it.

    I have used infinitest for this. It is a Continuous Testing plugin for Intellij/Eclipse. If you are doing javascript, i can also recommend grunt for Continuous Testing.


    to the comments

    About the „Is TDD Dead?“ discussion

    It did not take long after Agile died that TDD joined its party. If you havent noticed, David Heinemeier Hansson put up quite a rant against TDD on his blog leading to a lot of discussion - also on Twitter. Finally they ended up doing a live online discussion regularely featuring Kent Beck, Martin Fowler and David not (yet) involving the TDD advocate Robert C. Martin aka uncle bob. If you haven't watched it live, here is the initial 30min discussion of the series on Youtube. I really recommend the whole series.

    I think this topic and the ongoing discussion is very valueable to our craft because it is the kind of questioning and critical thinking that evolve what we do and how good we are at it. Even if we have a strong faith in something, we should not be afraid to question it, ever.

    I get davids points, however i do not entirely agree with all of his concerns. I want to share my thoughts in this blogpost.

    „Test induced design damage“
    A good design is a design that makes the software easy to change. I think this is something we all agree about. While TDD generally puts a lot of good pressure on the design, if you're not careful you're going to couple the tests tightly to the structure of your code. In this case even refactorings might force us to touch a lot of test-code aswell. You don't want that to happen. Tests should be the enabler of refactoring. I've happened to experience lots of applications that were so small that sociable unittests with reduced mocking and integration tests were just perfect. The value of this approach was that it was very easy to change/refactor production code without touching test-code. TDD does not solve these design problems, TDD does not design for you. TDD is just a design tool, but you are still the designer. Testing is engineering. Look for advantages, make tradeoffs. Trying to minimize mocking can definitely pay off.
    To Test first, or not to Test first
    The point is, you do not want to verify if the code is working manually. You'd have to do it way too often, and it would just take too long. You want to be able to fiddle with the code, and do small changes frequently, without having to try everything out manually. You want to do it automated and the feedback should be instant. So i see no point in 1. writing code, 2. trying it out, and then 3. writing a test. Why not write the test in the first place? Also, we tend to be very flaccid on writing a test for something that we have found to be working, because we tend to become euphoric about our success quickly which makes us tired of digging in details later. Other than that, the actual writing of the test first forces us to think about the problem to an extent that makes us recognize hidden details. These details are better be known before we start writing the code.

    TDD is so widespread nowadays
    I was laughing when i noticed that there are actually people who believe that everyone does tdd nowadays. Sadly, TDD, or lets say developer testing is in fact unused and/or undiscovered in many places. I've happened to see it missed in programming education and also software companies. Sometimes, it was even unwelcome. If you were using any kind of developer testing in these places, you were a pioneer. There is still plenty of work to do on this topic.

    The bottom line is, automated developer testing is good. We need it. We were silly if we didn't.


    to the comments

    An Annotation Nightmare

    @XmlElementWrapper(name="orders")
    @XmlJavaTypeAdapter(OrderJaxbAdapter.class)
    @XmlElements({
    @XmlElement(name="order_2",type=Order2.class),
    @XmlElement(name="old_order",type=OldOrder.class)
    })
    @JsonIgnore
    @JsonProperty
    @NotNull
    @ManyToMany
    @Fetch(FetchMode.SUBSELECT)
    @JoinTable(
    name = "customer_order",
    joinColumns = {
    @JoinColumn(name = "customer_id", referencedColumnName = "id")
    },
    inverseJoinColumns = {
    @JoinColumn(name = "order_id", referencedColumnName = "id")
    }
    )
    private List orders;

    Wait. What? Is this really what we have come to? I can't even see the damn property under this bloat. How did this happen? Yeah ok - we had to get rid of the old xml configuration horror somehow. But this? This is even WORSE. This class is supposed to be a goddamn pojo with a bunch of properties. Short and concise, easy to read. I, as a reader of this cass, am not interested at all how the database table is joining customers to orders. I'm neither interested how its being serialized. This is just implementation details. Reading this class, i am living in an object world and i want to know what data and behaviour the object has. Not more, not less. I don't care about column names, fetchtypes or json serialization for the moment. And i don't want to read, change or recompile this class for the sake of a tablename change. I don't want to add another annotation for storing this entity in a mongoDB neither. The entity should not have responsibility for these details. We are not only violating the Single Responsibility Principle here, we are doing a f****** responsibility party.

    Ok ok, enough of the rage. How do we deal with this issue? Some duplicate the entity for various layers with different annotation purposes. They map the entity onto the next layers related entity using an automated mapper like Dozer. Some even write that mapping themselves. But this is by no means a solution. It is just replacing one code smell with another one: Duplication.

    So please, focus on frameworks that don't force you to clutter your code. jOOQ is a nice solution to map database records to entities without annotations. Also, hibernate allows you to define your mappings in XML.

    Private Field Injection


    @Inject
    private MyService myService

    This is used quite often, while it shouldn't even be possible. The myService field is private, thus it's inaccessible from outside the class. Nevertheless it's possible and people do it. In reality it is a hack. The DI-framework sets the field using reflections doing setAccessible(true). You don't want hacks in your code, do you? Lets have a look at the alternatives:

    Setter Injection
    Well, at least its better than the private field injection, since its using a public method instead of hacking a private field. But still, ask yourself this: 'Is this class even supposed to live without the injected value?' Because if it's not, there is no reason what so ever for the class to get constructed without an instance of MyService. You want to implement this constraint on the class level and inside the constructor, not on the framework level.

    Constructor Injection
    This is usually the way to go. It allows you to
    • make the field immutable (there is usually no need to change it).
    • implement the constraint, that the class is not instantiatable without a given MyService in the right place.
    Of course it means that you cannot inject by annotation. But why would you want to? The class doesn't need to know, if it gets injections by a DI-Container or a Factory Class. It should not know anything of this. No @Autowired, no @Qualifier. All it needs to know is its own behaviour. Everything else should be handled outside of the class.
    One could use a configuration class or file for the actual injection.

    A DI-Container is a usefull tool that helps you wire your classes together. Use it for this purpose, but don't let it dictate your code. Uncle Bob wrote a great post where he explained how to use DI-Frameworks without having them dictate your code.

    @RunWith(SpringJUnit4ClassRunner.class) in UnitTests

    Why would you need this in unittests? Because it is automatically generated by your IDE/app template? No! You want to test the behaviour of a class, living in isolation in unittests. Not if the DI-Conainer is injecting a fields accordingly. Just inject yourself in a setup method. No DI-Container needed. By the way, all this testrunner does is these 3 lines of code.

    private TestContextManager testContextManager;
    //..
    this.testContextManager = new TestContextManager(getClass());
    this.testContextManager.prepareTestInstance(this);

    They are not worth blocking your only TestRunner slot. You want to keep it free for something like parameterized @RunWith(JUnitParamsRunner.class) or concurrency @RunWith(ConcurrentJunitRunner.class) tests.

    tl;dr
    Annotations have become more harmful than helpful these days. We should get back to pojos and focus on keeping our code as clutterless and framework-agnostic as possible to make it more readable and reuseable. Don't let frameworks dictate your codebases, since they should be exchangable tools. Beware of what a class should know, and what not. Some annotations are useful, most aren't.

    @DevNull({
     @SuppressWarnings
     @Autowired, 
    @Inject, 
     @Override 
     @XmlElementWrapper,
    @XmlJavaTypeAdapter,
    @XmlElement,
    @JsonIgnore,
    @JsonProperty,
    @ManyToMany,
    @Fetch,
    @JoinTable
    })


    to the comments

    Sessions, a Pitfall.

    I got asked quite often, why one shouldn't use sessions and whats so bad about them. In this post i want to sum up my thoughts and experiences.

    Sessions produce unnecessary complexity

    The term bugfoot is a combination of bug and bigfoot. It's used for a bug that appears only once and is not reproducable. A bugfoot tells you that something is wrong, and that it will probably come up again. But you cannot fix it, because you don't know the cause.

    If you want bugfoots in your software, just use sessions. Sooner or later you'll happen to see one.

    Sessions don't scale

    Lets say you are a professional chess player, and you'd like to play multiple people at the same time. If you'd try to remember every game and your strategy on it, you'll hit your capacity rather quick. Now imagine you were not remembering anything of those games, and you were just rereading the chessboard on every move. You could literally play 1.000.000 people at the same time, and it wouldn't make any difference to you.

    Now draw an analogy to your server. If your application gets a lot of load, you might have to distribute it to different servers. If you were using sessions, you'd suddenly had to replicate all sessions to all servers. The system would become even more complex and error prone.

    Sessionstate cannot be bookmarkable nor cachable

    Did you try to bookmark your shopping cart? Nah, of course you can't because it's empty as soon as the session runs out. Imagine a shop that actually allowed you to bookmark your cart. Like
    • http://example.com/carts/1337 or
    • http://example.com/cart?products=[{id:1, amount: 1}]
    How wonderful that would be.

    Conclusion

    You want a simple system that is easy to test and whose bugs are easy to reproduce. State leads to the opposite. HTTP was never ever ment to be stateful and it should have stayed this way.

    If you want some rest, follow these two rules
    • Keep all application state (not resource state) on the client
    • Keep all shared state (not session state) on the server


    to the comments

    Boost your development speed

    I'm really pragmatic on this. To go faster, you need to avoid the things that slow you down. These blockers may vary from developer to developer, but some are pretty common. In this post i will elaborate on the more common ones. Before i do so i want to remind you that to improve, we might need to unlearn in the first place.
    Tip: Sometimes, the greatest achievement in improving is to successfully unlearn.

    Are you using your Mouse?

    Don't let your mouse slow you down. If you are not using all of your fingers typing, you might want to stop reading here and go over to a typewriting course. The following recommendation also assumes that you are using a feature rich editor like vim or an ide. Every single time you move your hand over to your mouse for a click there is a keyboard shortcut for the exact action that you are not aware of, or not used to. You want to learn these. It's really not that hard,  you'll get used to it faster than you'd think. Just begin by putting your mouse over to the other (or wrong) side of your keyboard. You're now forcing yourself to think every time you want to click. When this happens, remember to think of a possible solution using your keyboard. Don't know any? Ask your friends or search the internet (yes, you can use your mouse for that). BUT: Do not use your mouse for what ever action you wanted to perform. After following these simple rules for a few days, you'll notice how irrelevant your mouse has become. If you start feeling comfortable using your keyboard only, you could even try some sessions with your mouse plugged out. Mastering your keyboard will significantly boost your development speed.
    Tip: You could print out a list of your IDEs shortcuts and use it as a cheatsheet

    Reduce the Time debugging

    Debugging costs an awful lot. It's not rare that we end up hunting a bug for hours or even days. Quite often we engage multiple co-workers to solve it. This is really not a scenario we want to be in, since its not only expensive, but also depressing. How can we avoid such a scenario? Pretty easy. Just separate the software into pieces that work in isolation, and write automated tests that cover the components behaviour. This way, we can easily distinguish the component the bug is actually sitting in, and the debugging will become a cinch. The better the tests, the easier to find the bug. However, you wont write good tests, if the system that you are testing is already done and working. You'll assume that there is no additional value of writing thoughtout tests, when it's already working. Overcome this dilemma by writing your tests BEFORE the implementation. As a side effect, this will also help you coding in many ways. You want to treat your tests as first class citizens and as what they are - at least as important as your implementation. Think of a blacksmith who's casting metal into a mold. What would be more important for him. The mold, or the first sculpture coming out of it? You know the answer. And guess what... the tests are your mold. I don't care what you call it. TDD, BDD or whatever. Just automate tests to cover your softwares behaviour, write them before the implementation, and treat them as what they are - at least as important as your implementation.
    Tip: When using TDD, you want to use a shortcut to quickly navigate between the test and its implementation. I've bound it to Alt+T


    Reduce the Time reading code

    I can't stress this enough: You spend more time reading code than writing code. Believe it or not, while implementing a new feature or solving a bug, you repeatedly read code. You read code before you start writing new code, you read code multiple times inbetween writing code, and you read code after you have written it. Reading a 50 line method is pretty time consuming. Thus you want to make reading as easy and smooth as possible. Write self explanatory code with good names and short methods. And Don't Repeat Yourself. Who wants to read anything twice? Invest some time on this, and it will pay off.

    Improve your search skills

    This is an important one. A searchengine has become the most important development tool. It is even more important than your ide. You can do anything without your ide if you can find out how, but you cannot do anything with your ide if you dont know how. You are probably googling regularely, so it's worth improving. Improve your searchterms. Anticipate helpful words like 'how to' or 'problem'. Try different variants and quote important words and sentences. If you keep getting bad results of a common subject or website, filter them out using a preceeding minus. Here are some examples:
    • while c -do -c++
    • SomeException -helpme.com
    • How to "clean code"
    If you keep getting out-dated information, limit your search to the last month/year. This can easily be done in the search options. On stackoverflow, do not always read the first post since its usually a question and code thats not working. Scroll down to the answers with the most upvotes. They are worth reading. Bookmark the sites/docs that you are using regularely. You don't want to search them over and over again. Also, consider code search engines (Examples here and here).


    Focus & Calm

    You want to get into and keep a good focus when working on a task. Get into your flow and do not allow disruptions. Several disruptions can easily cost you an hour or two each day. Use headphones in louder environments, and communicate that you'll be available for questions a little later. Focus requires a clear mind. Make sure you don't have to remember too much stuff. You don't want to be afraid of forgetting things. If you can feel the fear to forget, write that thing down - so you can forget. Also, don't work on too many different tasks/projects at the same time. One is perfect, two is ok. More is probably just slowing you down. I've happened to work on 7 different projects a single day. It's really really dragging from you. Also, stay calm even in the most stressful situation. Huddling just slows you down even more.

    Invest in Hardware

    Hardware is cheap, workhours are not. Good hardware will save lots of time in the long term. An SSD that makes you boot within seconds and loads everything real quick, saves at least 10 minutes a day which is 40 hours a year. Same goes with ram and cpu of course. Also, screensize really matters. Choose a large monitor to be able to see everything without resizing/scrolling windows/panels. I personally feel comfortable with ~27". You can speed up many tasks using two monitors also. Web development would be such a task for example.
    Tip: 16:9 is standard nowadays but not quite optimal for coding. The higher the screen, the more lines of code you see. Try a 16:10 instead.

    Optimize your Workflow, Question your Tools

    It depends on what you are working on, but the goal stays the same. You want the most lean workflow and you will constantly have to reevaluate and improve it. For example: If you are working on a Java Webapp, you want to be able to quickly get feedback without redeploying. You can do this by using a class-reloader like spring-loaded or JRebel, or use the Play Framework which does not require redeployments at all due to its architecture. You could also question the language. Is there something else than java that could be more appropriate? Automate everything: testing, building, deployment, notifications. Consider rapid prototyping if you are in the early stages of a project.



    to the comments

    Agility demands good Software Design

    From what i have seen, Agile Software Development is widely misinterpreted nowadays. Many people think to be agile, they just need a process like scrum to define a framework of sprints and iterations. They will sprint until they exhaust and change until they stagnate. But when you'd ask them what principles of objectoriented design are, they'd answer: "Aww yes, we use classes". And when you'd asked them if they did test driven development, they'd reply: "Ahh, this is the thing where you write tests first, isn't it?". Bottom line, people think to be agile without caring about good software design. But in the end, the most important thing is what nobody cares about. It is the agility of the software that they are producing. So if you want to be agile, a process and a mindset is not enough. You need to keep your software agile. By that i mean, you need to keep the design of the software as clean and flexible as possible, to be able to apply any future changes at minimal cost. Software design of course includes your source code. In the end, you are only as agile as your software is.

    Why Agile?

    A customer does not know what he wants in the first place. He might have a rough idea, but thats it. We cant even blame him for that, because the actual problem that we are supposed to solve is influenced by so many factors and opinions that it is almost impossible to define the best solution right away. Other than that, miscommunication will most likely make things worse. To solve these issues we start small, and deliver fast. Seeing the first result, the customer can verify if we are on track or not. He can check if we have understood, or if he himself made a bad call. He might recognize things that he hasnt thought of before. Delivering repeatedly and iteratively like that, our software will more likely be useful and accepted by the customer in the end. It will more likely lead our project to success. Of course we will have to deal with changes more often, which is why we have to make the software itself agile. In the end it is the software which needs to be changeable, not us.

    What it takes

    Not only does an agile development process put our software under frequent change, it also demands progress at a fast and steady pace. These facts can quickly cause our software to rot if we don't watch out. And when it does, we get slowed down even more and changes suddenly become impossible. So we'd better not forget about the most important part in agile software development: Keeping our software design as agile as possible, from high-level architecture to the very code itself. Sadly, this gets ignored way too often, and many agile projects fail because of it. They fail because they chose to go faster by neglecting quality, and they quickly rush into a deadend of code rot leading to stagnation. So please remember: It is the Software you are building that needs to be agile in order to adapt to change defined by the customer. It is the software that needs to be agile in order to react to change with minimal effort. Rushing is not gonna make your software agile. Rushing is not gonna make YOU agile. Agility comes hand in hand with quality. And high quality is not even slow, hell no. Good quality done right makes you go even faster.

    So what is Agile Software Design?

    Agile Software Design is what makes your software as clean and flexible as you need it to be. It demands deliberate adoption of discovered and well established design patterns and principles of objectoriented programming. This is not easy. In fact it's hard, it's very hard. Even if you are really good and experienced at it, you might fail at some point, having to rethink many of your previous design decisions. Agile Software Design is not something that you can rush. It needs time and deliberateness. But it will pay off in the long term by making you progress at a more constant pace, which will in fact be much faster than it would be without agile design. The graphic below should clarify how quick ignorant rushing can slow you down tremendously. The graphic is not based on statistics, but on experience.

    Agile Software Design does not end at high-level architecture. It goes on with coding. Bad code can quickly make your project more clumsy than a bad high-level design could. So the devil is in the details.

    What can you do to make your Software Design more Agile?

    From a Managers perspective? Trust your Developers, don't push them. Invest in experience and expertise.

    From a developers perspective? First of all, learn the principles and patterns of objectoriented programming - not just in theory. Being good at those needs lots of practice. There are plenty of books for this:
    and many more.

    Keep your code clean - don't underestimate this! When you are programming, you spend more time reading code, than writing code. Thus you want to minimize the time you have to read code by making it more readable. It's not a secret how clean code looks like. Take a look at Clean Code.
    Other than that, use the right practices like

    Test Driven Development  

    Just do it already. And if you don't know how, go on and start learning. I have heard numerous opinions of TDD where people said
    • "Unittests are too expensive. The Customer wont pay for that."
    • "Unittests slow us down, we needed to write more code in that time."
    • "I don't know. I'm pretty good at programming, Test-First feels needlessly hard and awkward."
    But these are just subjective opinions of people who are not experienced in TDD.
    TDD should in fact boost your longterm development speed if you are doing it right. I have not seen a single project in my life that has been as small and simple as that it would not have taken a significant profit from a TDD approach. And i have in fact seen very simple ones. The thing is, you have to test it anyways. What else would speed up your testing more than to automate it? Also, TDD significantly reduces the amount of time you have to spend debugging which already eats up lots of your time - at least more than you'd think. Other than that those tests
    • become a reliable documentation of your code,
    • allow you to fearlessly clean up your code,
    • force you to think of the problem before thinking of the actual solution, which will improve your initial draft,
    • enforce decoupling of components, which will make your design more agile.

    Pair Programming

    It's not only that 4 eyes will see more defects than 2 eyes do, but also that the transfer of knowledge is taking place. You want all developers to learn from each other to become equally valuable. And you always want more than one person to know a specific module to improve the truck factor. Heck yeah, some people even do mobprogramming - Seriously.

    Collective Code Ownership

    Don't allow one guy to own that modules code. All code on the project must belong to everybody. Everyone is allowed and invited to view, question and improve everyones code. This might be hard for some people, and might provoke conflicts, but it is absolutely mandatory to make the code better.

    Code Reviews

    Constantly review code changes. It is the only way to stay up to date, and it will reveal code smell. You need to reveal code smell, because if you don't, you will get stuck.

    Refactoring

    Refactor often. Everytime you see something that could be refactored, you need to do it right on the spot. The earlier you do it, the more time you save. So if you do it right then, you save the most possible amount of time.


    tl;dr: You are just as agile as your software design, which of course includes code. High speed does not demand low quality. It doesn't work like that. In fact high quality is needed to allow high speed. Agile design evolves high quality.


    to the comments

    spring-data-rest in Action

    What is spring-data-rest?

    spring-data-rest, a recent addition to the spring-data project, is a framework that helps you expose your entities directly as RESTful webservice endpoints. Unlike rails, grails or roo it does not generate any code achieving this goal. spring data-rest supports JPA, MongoDB, JSR-303 validation, HAL and many more. It is really innovative and lets you setup your RESTful webservice within minutes. In this example i'll give you a short overview of what spring-data-rest is capable of.

    Initial Configuration 

    I'm gonna use the new Servlet 3 Java Web Configuration instead of an ancient web.xml. Nothing really special here.

    public class WebAppInitializer extends AbstractAnnotationConfigDispatcherServletInitializer {

    @Override
    protected Class<?>[] getRootConfigClasses() {
    return new Class<?>[]{AppConfiguration.class};
    }

    @Override
    protected Class<?>[] getServletConfigClasses() {
    return new Class[]{WebConfiguration.class};
    }

    @Override
    protected String[] getServletMappings() {
    return new String[]{"/"};
    }
    }
    WebAppInitializer.java on github

    I am initializing hibernate as my database abstraction layer in the AppConfiguration class. I'm using an embedded database (hsql), since i want to keep this showcase simple stupid. Still, nothing special here.

    @Configuration
    @EnableJpaRepositories
    @EnableTransactionManagement
    public class AppConfiguration {

    @Bean
    public DataSource dataSource() {
    EmbeddedDatabaseBuilder builder = new EmbeddedDatabaseBuilder();
    return builder.setType(EmbeddedDatabaseType.HSQL).build();
    }

    @Bean
    public LocalContainerEntityManagerFactoryBean entityManagerFactory() {
    HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
    vendorAdapter.setDatabase(Database.HSQL);
    vendorAdapter.setGenerateDdl(true);

    LocalContainerEntityManagerFactoryBean factory = new LocalContainerEntityManagerFactoryBean();
    factory.setJpaVendorAdapter(vendorAdapter);
    factory.setPackagesToScan(getClass().getPackage().getName());
    factory.setDataSource(dataSource());

    return factory;
    }

    @Bean
    public PlatformTransactionManager transactionManager() {
    return new JpaTransactionManager();
    }
    }
    AppConfiguration.java on github

    Now to the application servlet configuration: WebConfiguration

    @Configuration
    public class WebConfiguration extends RepositoryRestMvcConfiguration {
    }
    WebConfiguration.java on github

    Oh, well thats a bit short isnt it? Not a single line of code required for a complete setup. This is a really nice application of the convention over configuration paradigm. We can now start creating spring-data-jpa repositories and they will be exposed as RESTful resources automatically. And we can still add custom configuration to the WebConfiguration class later if needed.

    The Initialization was really short and easy. We didn't have to code anything special. The only thing we did was setting up a database connection and hibernate, which is obviously inevitable. Now, that we have setup our "REST Servlet" and persistence, lets move on to the application itself, starting with the model.

    The Model

    I'll keep it really simple creating only two related entities.
    @Entity
    public class Book {

    @Id
    private String isbn;

    private String title;

    private String language;

    @ManyToMany
    private List<Author> authors;

    }
    Book.java on github

    @Entity
    public class Author {

    @Id
    @GeneratedValue(strategy = GenerationType.SEQUENCE)
    private Integer id;

    private String name;

    @ManyToMany(mappedBy = "authors")
    private List<Book> books;

    }
    Author.java on github

    To finally make the entities persistent and exposed as a RESTful webservice, we need spring-data repositories. A repository is basically a DAO. It offers CRUD functionality for our entities. spring-data takes away most of your programming effort creating such repositories. We just have to define an empty interface, spring-data does everything else out of the box. Still, it is easily customizable thanks to its design by convention over configuration!

    The Actual Repositories


    @RestResource(path = "books", rel = "books")
    public interface BookRepository extends PagingAndSortingRepository<Book, Long> {
    }
    BookRepository.java on github


    @RestResource(path = "authors", rel = "authors")
    public interface AuthorRepository extends PagingAndSortingRepository<Author, Integer> {
    }
    AuthorRepository.java on github

    Again, there is nearly no code needed. Even the @RestResource annotation could be left out. But if i did, the path and rel would be named after the entity, which i dont want. A REST resource that contains multiple children should be named plural though.

    Accessing The Result

    Our RESTful webservice is now ready for deployment. Once run, it lists all available resources on the root, so you can navigate from there.

    GET http://localhost:8080/
    {
    "links" : [ {
    "rel" : "books",
    "href" : "http://localhost:8080/books"
    }, {
    "rel" : "authors",
    "href" : "http://localhost:8080/authors"
    } ],
    "content" : [ ]
    }

    Fine! Now lets create an author and a book.

    POST http://localhost:8080/authors
    {"name":"Uncle Bob"}

    Response
    201 Created
    Location: http://localhost:8080/authors/1
    PUT http://localhost:8080/books/0132350882
    {
    "title": "Clean Code",
    "authors": [
    {
    "rel": "authors",
    "href": "http://localhost:8080/authors/1"
    }
    ]
    }

    Response
    201 Created
    Noticed how i used PUT to create the book? This is because its id is the actual isbn. I have to tell the server which isbn to use since he cant guess it. I used POST for the author as his id is just an incremental number that is generated automatically. Also, i used a link to connect both, the book (/books/0132350882) and the author (/authors/1). This is basically what hypermedia is all about: Links are used for navigation and relations between entities.

    Now, lets see if the book was created accordingly.

    GET http://localhost:8080/books
    {
    "links" : [ ],
    "content" : [ {
    "links" : [ {
    "rel" : "books.Book.authors",
    "href" : "http://localhost:8080/books/0132350882/authors"
    }, {
    "rel" : "self",
    "href" : "http://localhost:8080/books/0132350882"
    } ],
    "title" : "Clean Code"
    } ],
    "page" : {
    "size" : 20,
    "totalElements" : 1,
    "totalPages" : 1,
    "number" : 1
    }
    }

    Fine!

    Here is an Integration Test, following these steps automatically. It is also available in the example on github.

    public class BookApiIT {

    private final RestTemplate restTemplate = new RestTemplate();

    private final String authorsUrl = "http://localhost:8080/authors";
    private final String booksUrl = "http://localhost:8080/books";

    @Test
    public void testCreateBookWithAuthor() throws Exception {
    final URI authorUri = restTemplate.postForLocation(authorsUrl, sampleAuthor()); // create Author

    final URI bookUri = new URI(booksUrl + "/" + sampleBookIsbn);
    restTemplate.put(bookUri, sampleBook(authorUri.toString())); // create Book linked to Author

    Resource<Book> book = getBook(bookUri);
    assertNotNull(book);

    final URI authorsOfBookUri = new URI(book.getLink("books.Book.authors").getHref());
    Resource<List<Resource<Author>>> authors = getAuthors(authorsOfBookUri);
    assertNotNull(authors.getContent());
    assertFalse(authors.getContent().isEmpty()); // check if /books/0132350882/authors contains an author
    }

    private String sampleAuthor() {
    return "{\"name\":\"Robert C. Martin\"}";
    }

    private final String sampleBookIsbn = "0132350882";

    private String sampleBook(String authorUrl) {
    return "{\"title\":\"Clean Code\",\"authors\":[{\"rel\":\"authors\",\"href\":\"" + authorUrl + "\"}]}";
    }

    private Resource<Book> getBook(URI uri) {
    return restTemplate.exchange(uri, HttpMethod.GET, null, new ParameterizedTypeReference<Resource<Book>>() {
    }).getBody();
    }

    private Resource<List<Resource<Author>>> getAuthors(URI uri) {
    return restTemplate.exchange(uri, HttpMethod.GET, null, new ParameterizedTypeReference<Resource<List<Resource<Author>>>>() {
    }).getBody();
    }
    }
    BookApiIT.java on github

    Conclusion

    We have created a complete RESTful webservice without much coding effort. We just defined our entities and database connection. spring-data-rest stated that everything else is just boilerplate, and i agree.

    To consume the webservices manually, consider the rest-shell. It is a command-shell making the navigation in your webservice as easy and fun as it could be. Here is a screenshot:


    The complete example is available on my github
    https://github.com/gregorriegler/babdev-spring/tree/master/spring-data-rest
    https://github.com/gregorriegler/babdev-spring



    to the comments

    Good Bye Redeployment. spring-loaded, an Opensource Classreloader

    Redeem yourself from hot deployments and OutOfMemoryException. spring-loaded is an opensource classreloader and a promising alternative to JRebel. It does not offer as many features as JRebel does, and it does not support any framework just yet. Nevertheless, its a great tool that can save the time you are waiting for that servletcontainer to restart again.

    What it can do:
    • add/modify/delete methods/fields/constructors
    • modify annotations on types/methods/fields/constructors
    • add/remove/change values in enum types
    What it cant:
    • support framework specific changes like Spring MVC @RequestMappings
    • change log configuration on the fly
    It is not getting the attention that it deserves, so give it a try.

    Just download from github already, and add the following to your JVM parameters:
    java -javaagent:<pathTo>/springloaded-{VERSION}.jar -noverify


    to the comments

    Why REST is so important

    This post is dedicated to REST, an architectural style of shaping webservices and one of the most misunderstood concept in the history of IT. This post is addressed to you who is designing webservice apis not being fully aware what REST actually means. I'm trying to give you the idea. This post is also addressed to you who think to know what REST means, when in reality you have no clue, just yet. Yes i have met such people in the past - plenty of them. It's not going into the details of the Richardson Maturity Model, and it's not gonna make you a REST expert. There are plenty of guides on the web for that: slides, youtube videos, blogposts, books and more. Rather than going into the details, i'm going to link some good resources at the end of this post.

    So lets start with

    The meaning of REST

    Representational State Transfer. This sentence is not only what REST stands for, it is also the tiniest possible description of what REST actually means. Didn't get it? Read it again: Representational State Transfer. It is not a standard, rather a style describing the act of transfering a state of something by its representation.

    Lets consider this:
    Marcus is a farmer. He has a ranch with 4 pigs, 12 chickens and 3 cows. He is now simulating a REST api while i am the client. If i want to request the current state of his farm using REST i just ask him: "State?"
    Marcus answers: "4 pigs, 12 chickens, 3 cows".
    This is the most simple example of Representional State Transfer. Marcus transfered the state of his farm to me using a representation. The representation of the farm is the plain sentence: "4 pigs, 12 chickens, 3 cows".

    So lets get to the next level. How would i tell Marcus to add 2 cows to his farm the REST way?
    Maybe tell him: "Marcus, please add 2 cows to your farm".
    Do you think this was REST? Are we transfering state by its representation here? NO! This was calling a remote procedure. The procedure of adding 2 cows to the farm.
    Marcus sadly answers: "400, Bad Request. What do you mean?"
    So lets try this again. How would we do this the REST way? What was the representation again? It was "4 pigs, 12 chickens, 3 cows". Ok. so lets try this again transfering the representation...
    me: "Marcus, ... 4 pigs, 12 chickens, 5 cows ... please!".
    Marcus: "Alright !".
    me: "Marcus, ... what is your state now?".
    Marcus: "4 pigs, 12 chickens, 5 cows".
    me: "Ahh, great!"
    See? It was really not that hard and it was REST.

    Why RPC is a pain in the A**

    So why would you favor REST over the remote procedure call (=RPC) from a logical perspective? Because it dramatically reduces the complexity of our communication by making the representation our only contract. We do not have to discuss what kinds of procedures we need (Add a cow?, Add an animal of a type? Double the chickens amount? Remove all pigs?). All we have to discuss is the representation, and use this representation to achieve anything we want. Easy, isn't it? The needless complexity of RPC is not helpful at all. It is rather increasing the risk of misunderstandings, which we don't want. We don't want our communication to fail because Marcus and I understood a procedure differently.
    But this is just one of many problems RPC is creating. If you want to use RPC you need to design some kind of structure to embed your procedure into. This structure requires a place to store parameters, error codes, return values and so on. I have seen lots of developers and companies who really did this. They designed their own RPC-Structure arising huge problems in the implementation of clients and client-server interaction. Why would you do this? Why would you invent your own RPC-Structure? Do you think this is helpful? What if i wanted to make an application that uses many WebServices of multiple proprietary RPC-Formats? I would have to develop something like this:
    image #1

    Ugh...
    If you really need RPC, at least choose a standard like SOAP. Don't make up your own stuff!

    But SOAP is still bad

    Still, even the standards of RPC are really painful. Well, i have to admit that with ACID Transactions, and a complete standardized Service Description Language, SOAP is not all that bad in some circumstances. Nevertheless, the overhead SOAP produces is massive and a huge performance killer. HTTP is a lightweight protocol. Its headers include anything you need. The only thing you want to put in the body is a representation - or not even that.

    Sessions are Evil

    Since i wrote solely about sessions and why they are bad in another post, i removed this paragraph. Please continue reading here: Sessions. A Pitfall

    Dont reinvent Hypermedia

    Since hypermedia is getting quite popular now, i beg you: Don't invent your own style.
    We do already have plenty. There is
    And we are slowly getting to the situation in image #1 again.

    Further Resources

    In this post, i have only scratched the surface of the advantages of REST.
    Here are some good resources to get you a deeper understanding.


    to the comments

    Spring MVC - @RequestBody and @ResponseBody demystified

    In this post i want to dig into spring mvc a little, revealing what happens behind the scenes when a request is converted to your parameter object and vice versa. Before we start, i want to explain the purpose of these annotations.

    What are @RequestBody and @ResponseBody for?


    They are annotations of the spring mvc framework and can be used in a controller to implement smart object serialization and deserialization. They help you avoid boilerplate code by extracting the logic of messageconversion and making it an aspect. Other than that they help you support multiple formats for a single REST resource without duplication of code. If you annotate a method with @ResponseBody, spring will try to convert its return value and write it to the http response automatically. If you annotate a methods parameter with @RequestBody, spring will try to convert the content of the incoming request body to your parameter object on the fly.

    Here is an example

    @Controller
    @RequestMapping(value = "/bookcase")
    public class BookCaseController {

    private BookCase bookCase;

    @RequestMapping(method = RequestMethod.GET)
    @ResponseBody
    public BookCase getBookCase() {
    return this.bookCase;
    }

    @RequestMapping(method = RequestMethod.PUT)
    @ResponseStatus(HttpStatus.NO_CONTENT)
    public void setBookCase(@RequestBody BookCase bookCase) {
    this.bookCase = bookCase;
    }

    }

    So what is Spring doing behind the scenes when we are using those Annotations?


    Depending on your configuration, spring has a list of HttpMessageConverters registered in the background. A HttpMessageConverters responsibility is to convert the request body to a specific class and back to the response body again, depending on a predefined mime type. Every time an issued request is hitting a @RequestBody or @ResponseBody annotation spring loops through all registered HttpMessageConverters seeking for the first that fits the given mime type and class and then uses it for the actual conversion.

    How can i add a custom HttpMessageConverter?


    By adding @EnableWebMvc respectively <mvc:annotation-driven />, spring registers a bunch of predefined messageconverters for JSON/XML and so on. You can add a custom converter like the following

    @Configuration
    @EnableWebMvc
    @ComponentScan
    public class WebConfiguration extends WebMvcConfigurerAdapter {

    @Override
    public void configureMessageConverters(List<HttpMessageConverter<?>> httpMessageConverters) {
    httpMessageConverters.add(new BookCaseMessageConverter(new MediaType("text", "csv")));
    }
    }

    In this example i've written a converter that handles the conversion of a BookCase, which is basically a List of Books. The converter is able to convert csv content to a BookCase and vice versa. I used opencsv to parse the text.

    Here is the model

    public class Book {

    private String isbn;

    private String title;

    public Book(String isbn, String title) {
    this.isbn = isbn;
    this.title = title;
    }

    // ...
    }

    public class BookCase extends ArrayList<Book> {

    public BookCase() {
    }

    public BookCase(Collection<? extends Book> c) {
    super(c);
    }
    }

    and the actual converter

    public class BookCaseMessageConverter extends AbstractHttpMessageConverter<BookCase> {

    public BookCaseMessageConverter() {
    }

    public BookCaseMessageConverter(MediaType supportedMediaType) {
    super(supportedMediaType);
    }

    public BookCaseMessageConverter(MediaType... supportedMediaTypes) {
    super(supportedMediaTypes);
    }

    @Override
    protected boolean supports(Class<?> clazz) {
    return BookCase.class.equals(clazz);
    }

    @Override
    protected BookCase readInternal(Class<? extends BookCase> clazz, HttpInputMessage httpInputMessage) throws IOException, HttpMessageNotReadableException {
    CSVReader reader = new CSVReader(new InputStreamReader(httpInputMessage.getBody()));
    List<String[]> rows = reader.readAll();
    BookCase bookCase = new BookCase();
    for (String[] row : rows) {
    bookCase.add(new Book(row[0], row[1]));
    }
    return bookCase;
    }

    @Override
    protected void writeInternal(BookCase books, HttpOutputMessage httpOutputMessage) throws IOException, HttpMessageNotWritableException {
    CSVWriter writer = new CSVWriter(new OutputStreamWriter(httpOutputMessage.getBody()));
    for (Book book : books) {
    writer.writeNext(new String[]{book.getIsbn(), book.getTitle()});
    }
    writer.close();
    }
    }

    The Result 

    We can now issue text/csv requests to our Resource along with application/json and xml which are basically supported out of the box.

    1.)
    PUT /bookcase
    Content-Type: text/csv
    "123","Spring in Action"
    "456","Clean Code"


    Response
    204 No Content
    2.)
    GET /bookcase
    Accept: text/csv

    Response
    200 OK
    "123","Spring in Action"
    "456","Clean Code"


    Thanks to the design of spring mvc, which is following the single responsibility principle, our controller stays thin. We don't have to add a single line if we want to support new media types.

    The complete example is available on my github


    to the comments