Paginator for those suffering from PostgreSQL count(*) speed issues

Our company has been using PostgreSQL for a very long time and has found it a very solid product for our needs. One thing that we've run into is that as our database grows (1mil+ records), there is a major slowdown on page loads due to a PostgreSQL bug. To resolve this, I had to re-create a new paginator that didn't rely on the count(*) sql call. I changed it to work similar to how Google performs their queries relying solely on the offset and max to determine which pagination buttons to show.

Here's my final taglib:

Grails Screencasts

Here are several screencasts that I did for TeachMeToCode.com

Introduction to Grails – Part 1

Are you new to Grails? Grails is an excellent alternative to Ruby on Rails, which uses Groovy / Java. In this screencast, learn how to create a simple CRUD system with Grails.

Introduction to Grails – Part 2

In this screencast, we continue on our task of creating a ‘Stack Overflow’ clone by adding more domain classes and relationships between them. We also cover how to test your classes using integration tests.

Skinning in Grails

In this screencast, I cover how to use custom Grails configurations along with filters to enable url-based site skins.

Interview with Graeme Rocher on Grails 1.4 and 2.0

In case you missed the interview done by ThirstyHead here it is.

Great stuff, I personally can't wait to see what the future holds for Grails!

Roll your own Ajax-Based Captcha in Grails

Recently, I was asked to come up with a better solution to our captcha needs. We have been using ReCaptcha, which is great but difficult to read at times, and has caused frustrated customers and lost sales. I found a great solution at http://www.jkdesign.org/captcha which displays a number of graphics and let's the user choose the right one to prove they are human. Here is a screenshot of my implementation:

To make this work within Grails, I had to make several tweaks. The following files are required:

• JQuery 1.2+ (I am using version - 1.4.2)
• JQuery UI (I am using version - 1.8.2)
• jquery.simpleCaptcha-0.2.js
• Captcha Images placed in images/captchaImages
• BCrypt.java by Damien Miller
• CaptchaController.groovy (below)

Create a new controller called Captcha. This can really be named anything, but if you do rename it, it will have to be updated in the jquery.simpleCaptcha-0.2.js file or passed in as an option via the javascript.

What this controller does is return a JSON object with the data needed to generate the captcha. The JSON appears like so:

Now we just need to implement this in our GSP file and controller. Suppose we have a page like shown above with a pickup code and the last 4 digits of the persons phone number. With adding our captcha div and required javascript, our GSP would look like this:

Finally, we need to perform the validation on the controller side. The modified authentication action would look like the following:

So there ya go. It's actually pretty easy and customers seem to like choosing an image much more than typing a word that is difficult to read.

Uploading Files in Grails

Here's the slides from a presentation I did at the UGGUG in May 2010. I found that I needed to use it again today so I posted it.

Upload files with grails

View more presentations from Eric Berry.

Here is the source code for the uploader service I created:

A tip that you may enjoy is the way I implemented this uploader into a real world application. I wanted to be able to upload files into a folder within the web-app folder for development, but in production, I wanted to put it in a folder that is served via Tomcat directly. So on our production server, I created a folder at /opt/assets and created a symbolic link in the $TOMCAT_ROOT/webapps/assets to point to it. I modified the service as follows:

I then create a folder in my web-app folder named 'assets', being sure to add it to my ignore list for the repository. So once I upload the file, it will save to the correct location.

Now to view the image, all I have to do is the following:

This now will work in both DEVELOPMENT and PRODUCTION enviroments.

http://github.com/cavneb/FileUploader

Passing Data From View to Layout via pageProperty

I am writing a new Grails application which uses the website template Admintasia. Part of the layout gsp file has a section for the header and sub-header. For this to be used, I needed to be able to pass those two strings from the view to the layout.

Here's what my view looks like:

And here's what my layout looks like:

I hope this helps out!

Using Tag Lib within Controllers and Services

So let's say you want to utilize the standard grails tag lib of formatNumber within your domain class, controller or service. How would you go about doing it? With the help of a blog post by Lucas Teixeira, I was able to get it working without an issue. Here is the code:

It's also simple to utilize your own tag libs as well. Here's a tag lib that I created to help with XML formatting:

So to call this tag lib in the domain class, controller or service, I just need to do the following:

Just make sure that you include the 'def grailsApplication' in the beginning.

Using Criteria Builder with Projections

Something I had to dig around for today was how to perform a sum on a table using Criteria Builder. It seems that it is treated a bit different than a normal query would be. Here is an example of what I tried and failed at:

After playing around with it a bit, I found that I can't use the 'and' closure. I revised the code to the following and it worked:

Simple Ehcache in Grails

The other day I needed to cache some results from a several remote services within my Grails application. It wasn't critical to that the response from the remote services be "real-time" and it was O.K. if the information was 10 minutes old (but no older). There are so many ways I could have done this within my Grails application, but I turned to EhCache since I was already using it for my Hibernate 2nd level cache.

First thing I needed to do was setup a simple cache. So here is what I added to resources.groovy:

beans = {
  simpleRemoteServiceCache(
    org.springframework.cache.ehcache.EhCacheFactoryBean) {
    timeToLive = 600 // life span in seconds
  }
}

Next it was as simple as injecting this new bean into my services and making use of the cache. Here is a quick example of checking the cache, getting from it, and adding to it.

def dataThatMightBeCached
if (simpleRemoteServiceCache.get("theCacheKey")) {
  dataThatMightBeCached = simpleRemoteServiceCache.get("theCacheKey").getValue()
} else {
  // ... other code to get the data to be cached ...
  simpleRemoteServiceCache.put(
    new net.sf.ehcache.Element("theCacheKey", 
      dataThatMightBeCached)
  )
}

That just about does it. EhCache handles expiring the content automatically based upon the timeToLive property for us. Pretty simple really. I could have configured the cache even further either in resources.groovy or in ehcache.xml, but the defaults work for my needs right now.

You can find similar examples of using caches on the Ehcache samples page.

Grails and simple realm-based authentication

Today's post is brought to you by skamster from the #grails IRC channel on freenode and describes how to setup memory-realm-based authentication with Tomcat.

First, you will need to "install templates" if you haven't already. Next, you will need to edit your web.xml file located in src/templates/war and replace the old security constraint (towards the end of the file) with the following:

<security-constraint>
<web-resource-collection>
<web-resource-name>
web
</web-resource-name>
<!-- This would protect the entire site -->
<url-pattern>/*</url-pattern>

</web-resource-collection>
<auth-constraint>
<!-- Roles that have access -->
<role-name>testRole</role-name>
</auth-constraint>
</security-constraint>

<!-- BASIC authentication -->
<login-config>
<auth-method>BASIC</auth-method>
<realm-name> Example Basic Authentication </realm-name>
</login-config>

The above is telling Tomcat that you want require basic authentication for all URLs and what Roles have access. Next, you will need to add a user and role.

The final step is to create a script called "_Events.groovy" in your scripts directory for your project with the following:

eventConfigureTomcat = {tomcat ->
  tomcat.addUser("testUser", "testpassword")
  tomcat.addRole("testUser", "testRole")
}

This simply adds a user (testUser) with the role of "testRole" to the authentication realm for Tomcat.

That's it. Now you have simple realm-based authentication setup for your application. Enjoy!

Grails JMX - Redux

Here is an updated resources.groovy with some additional JMX controls exposed. Thanks to schmolly159 on the #grails freenode IRC channel for the examples and suggestion!

beans = {
  // JMX mBean server
  mbeanServer(MBeanServerFactoryBean) {
    locateExistingServerIfPossible=true
  }

  // log4j JMX
  log4jBean(HierarchyDynamicMBean)

  // Hibernate stats
  hibernateStats(org.hibernate.jmx.StatisticsService) {
    sessionFactory = ref("sessionFactory")
    statisticsEnabled = true
  }
 
  // JMX exporter 
  exporter(MBeanExporter) {
    server = mbeanServer
    beans = [
    "log4j:hierarchy=default": log4jBean,
    "hibernate:name=HibernateStatistics": hibernateStats,
    "dataSource:name=DataSource": ref("dataSource"),
    "quartz:name=QuartzScheduler": ref("quartzScheduler")
    ]
  }
}

A few things to note about the above configuration.

First, the "statisticsEnabled" property of the hibernate statistics bean must be after the sessionFactory otherwise the sessionFactory will reset this to the default of "false".

Second, the above also assumes you have the Quartz plug-in installed.

Otherwise, it's that simple.

Do you have any beans that you find yourself exposing through JMX? Let us know in the comments!

Grails log4j JMX in one easy step

Here is a quick way to enable log4j control in JMX with Grails. Just update your resources.groovy like so:

import org.springframework.jmx.support.MBeanServerFactoryBean
import org.springframework.jmx.export.MBeanExporter
import org.apache.log4j.jmx.HierarchyDynamicMBean

beans = {
  log4jBean(HierarchyDynamicMBean)
 
  mbeanServer(MBeanServerFactoryBean) {
    locateExistingServerIfPossible=true
  }
 
  exporter(MBeanExporter) {
    server = mbeanServer
    beans = ["log4j:hierarchy=default":log4jBean]
  }
}

That's it. Now you can use jconsole to control your log4j settings, which during development/debug can be quite helpful. Of course you could take this much further by only enabling it in specific environments (e.g. development only), but that's up to you!

Groovy and Grails Strings

The other day on the freenode #grails channel schmolly159 pointed out a subtle diffrence in Groovy (and thus Grails) string handling. Let's see an example first.

log.warn("I am warning you!")
log.warn('I am warning you!')

In this example there are two log warning statements. They look very similar don't they? Look closely at the very minor difference. One is using double quotes for the string value and the other is using single quotes.

Now, the interesting part is actually how Groovy interprets the string. The double quoted string is actually a GString while the single quoted string is just a Java string.

"So what?", you might ask. In this specific example there is no technical reason to make Groovy use a GString. It's not using any of the features of the GString and thus we are just making Groovy work a bit harder to give us a Java string. Regardless of what "a bit harder" actually equates to (nanoseconds most likely), it all adds up.

In very large systems things like this add up over time. So be thoughtful about what kind of strings you are using. It could make a difference in the long-run.

Integration Tests with Shiro and Nimble

Today we have a guest posting from Uris on the freenode.net #grails channel. I know he spent quite a bit of time trying to solve this issue so rather than keep it to himself he wanted to share it with everyone else.

I recently hit a road block when trying to do integration tests for some controllers that required an authenticated user. We were using Nimble plugin, which basically uses shiro for authentication. Nimble injects an instance of the authenticated user into each controller called 'authenticatedUser'. The particular controller we were testing required that an 'authenticatedUser' be passed on to a service to do a certain action. I kept getting the following error when running the test:

No SecurityManager accessible to the calling code, either bound to the 
org.apache.shiro.util.ThreadContext or as a vm static singleton. 
This is an invalid application configuration. 
java.lang.IllegalStateException: No SecurityManager accessible to the 
calling code, either bound to the org.apache.shiro.util.ThreadContext 
or as a vm static singleton. 
This is an invalid application configuration.
at org.apache.shiro.SecurityUtils.getSecurityManager(SecurityUtils.java:115) 
at org.apache.shiro.SecurityUtils.getSubject(SecurityUtils.java:57)
at org.apache.shiro.SecurityUtils$getSubject.call(Unknown Source)
at NimbleGrailsPlugin$_injectAuthn_closure8.doCall(NimbleGrailsPlugin.groovy:137)
at NimbleGrailsPlugin$_injectAuthn_closure8.doCall(NimbleGrailsPlugin.groovy)

I needed to find a way to mock an authenticated user into the test for the controller to use. After a lot of googling, volumes of java, and lots of Iron Maiden, we finally discovered how to do it. First we needed to mock an instance of org.apache.shiro.subject.Subject:

def subject = [getPrincipal: {863},
  isAuthenticated: {true}
]as Subject

863 is the id of the default admin user created by Nimble during bootstrap. It might differ for another application, but in this particular case it is 863. Next we need to inject this Subject into org.apache.shiro.util.ThreadContext:

ThreadContext.put(ThreadContext.SECURITY_MANAGER_KEY,
  [getSubject: {subject} as SecurityManager])

The final step is to make sure that Shiro's SecurityUtils returns an instance of Subject:

SecurityUtils.metaClass.static.getSubject = {subject}

The final code looks something like this:

import org.apache.shiro.util.ThreadContext
import org.apache.shiro.subject.Subject
import org.apache.shiro.SecurityUtils

def subject = [getPrincipal: {863},
  isAuthenticated: {true}
]as Subject
ThreadContext.put(ThreadContext.SECURITY_MANAGER_KEY,
  [getSubject: {subject} as SecurityManager])
SecurityUtils.metaClass.static.getSubject = {subject}

Firefox Add-ons

Working with Grails and JSON/XML data is quite simple. However, during development sometimes it's nice to see what's being sent back and forth between the browser and the server. That's where these two Add-ons for FireFox come in.

1. HTTPFox - This little add-on will give you the ability to see the HTTP headers, posted data, query string data, and raw response (including XML and JSON). Always a handy little tool to have.

2. JSONView - While HTTPFox will show you the raw JSON in the response, this little add-on will show you the JSON in a formatted (pretty) way. Making debugging a bit easier.

These two add-ons have been a real life saver many of times for me when digging around trying to find that one little thing that is wrong. They might just help you too!

Testing Grails UrlMappings

One thing that often gets overlooked in testing Grails applications is testing any changes you may have made to UrlMappings.groovy. It's understandable that this would happen, but here is how to fix that.

Let's say you have the following in your UrlMappings.groovy

class UrlMappings {
  static mappings = {
    "/login"(controller: "security", action:"login")
    "/$controller/$action?/$id?"{
        constraints {
          // apply constraints here
        }
    }
    "/"(view:"/index")
    "500"(view:'/error')
  }
}

Really nothing much going on here with the one exception of the custom mapping of /login to the security controller login action. So that's really what we want to test, and here is how:

class UrlMappingsTestCase extends GrailsUrlMappingsTestCase {
  void testMappings() {
    assertUrlMapping("/login", 
      controller: "security", 
      action: "login"
    )

    assertForwardUrlMapping(500, view: "error")
  }
}

That's the basics. Nothing too complicated going on here. I even threw in an example of testing status code (500 in this case) and forwarding. GrailsUrlMappingsTestCase has other asserts that can be used to test other types of mappings so you might want to to read the source.

Remember, as a rule of thumb: "If it's important enough to write, it's important enough to test."

Groovy ranges and combinations

Today I stumbled upon Groovy combinations and thought I would share my findings. The idea behind them is quite simple. Take a collection of collections and produce all the combinations of those items. Here is a very quick example:

def stuff = [['apple', 'orange'], ['cat', 'zebra']]
def combo = stuff.combinations()
println combo

// results in: [[apple, cat], [orange, cat], 
// [apple, zebra], [orange, zebra]]

That's neat, and useful, but really not exciting.

That's where ranges come in to play. Remember, ranges are collections too! So we can have something like this:

def pat = { "${one}-${two}-${three}" }

def range1 = 1..2
def range2 = 'A'..'B'
def range3 = 'X'..'Z'
def combo = [range1, range2, range3].combinations()

def applied = combo.collect {
    one = it[0]
    two = it[1]
    three = it[2]
    pat()
}
println applied

// results in: [1-A-X, 2-A-X, 1-B-X, 2-B-X, 1-A-Y, 
// 2-A-Y, 1-B-Y, 2-B-Y, 1-A-Z, 2-A-Z, 1-B-Z, 2-B-Z]

Now that's exciting!

Grails partial validation

So yesterday I pointed out how partial binding could be done with your domain classes. Today, I will point out how partial validation of your domain classes can be achieved. Let's change our domain class from yesterday to have some constraints.

package com.xyzcorp

class Widget {
  Date dateCreated
  String sku
  int quantity

  static constraints = {
    dateCreated(nullable: false)
 sku(nullable: false, blank: false, maxSize: 15)
 quantity(nullable: false, min: 0)
  }

  static mapping = {
    autoTimestamp true
  }
}

Even though the nullable constraints are true by default for all properties on a domain class, I find it helpful to actually include them so there is no confusion about the intent. Given the same situation as yesterday where we only want to bind the sku property we will only validate the sku property with partial validation. Here is an example of how that is accomplished:

  // some code within our controller or service
  // instead of calling widgetInstance.validate() we can do the following
  widgetInstance.validate(['sku'])

What we are doing here is passing a map of properties to validate telling it specifically which properties to validate. As you can see this isn't a difficult thing to do, it's just one of those things that you may not have known about.

Using partial validation in combination with partial binding Grails gives you the tools to avoid being forced to use command objects. Another use of partial validation is to avoid expensive validation of properties that you know haven't changed. Also, this is also very useful when you need to test the validation implementation for a specific property in a unit/integration test.

Questions? Comments? Suggestions? Let me know!

Run last failed test, again!

In the course of fixing an issue or implementing a new feature you might just find yourself running your "grails test-app" over and over. Yes, you could also supply a pattern matcher to limit the things to test, but even better is the ability to just run the last failed tests using "grails test-app -rerun".

Grails partial binding

Ever have a domain class that only needs some of the properties set in a controller, and you want to make sure that specific parameters never get set? Here is a quick example of how you can do that.

Let's say we have the following domain class

package com.xyzcorp

class Widget {
  Date dateCreated
  String sku
  int quantity = 0

  static mapping = {
    autoTimestamp true
  }
}

Now imagine we have a controller where we are creating the widget and the dateCreated or quantity properties should never be set by the UI (since our system is maintaining those internally). Here is how:

  // some code in our controller
  Widget widget = new Widget()
  bindData(widget, params, [
    exclude: [
      'dateCreated', 
      'quantity'
    ]
  ])

As you can see we are using bindData to bind the params presented to our controller to our widget instance while excluding the dateCreated and quantity properties.

You could argue that we should just set the sku property in the constructor, but on larger domain classes it can be easier to exclude properties instead. Plus, our intent is pretty clear for future developers who are maintaining our code that we don't ever want dateCreated or quantity to be set from the UI. A big plus if you ask me!

This is just an example of excluding data from binding and bindData has quite a few other options besides exclude, so I encourage you to explore them.

Groovy and Grails code snippets

Sometimes the best way to learn new things is to read other peoples code. I personally learned quite a lot reading the Spring framework and Spring security source code over the years, but that's not for the faint of heart.

Snipplr gives you the opportunity to browse smaller code snippets and maybe even pick up a few new tricks. Searching around I have found some gems in the Groovy and Grails categories.

Here are a few I found this morning:

1. Download files from FTP in Groovy using Ant task
   
2. Groovy way to create File and URL MD5 hashes

What cool snippets did you find? Let's hear about them!

Code quality

Creating maintainable Grails applications isn't too tough. The convention over configuration approach of Grails helps address a lot of the problems suffered by larger applications in the maintenance phase. It's easy to find where bug fixes, changes, or enhancements should be applied.

However, this doesn't mean there is room for improvement. It's not a fundamental issue with Grails but applications in general. Having a proper set of test suites which cover the vast majority of the applications functionality goes a long way. That's why I have found the coverage plug-in so useful.

The coverage plug-in can help you identify areas within your application where your test coverage is lacking. This might even lead you to find out that the code is difficult to test and possibly designed incorrectly. I have found that if something is hard to test, then it's probably done wrong.

Along with the coverage plug-in, the codenarc plug-in can help you identify issues (some very minor) that will help you build more maintainable Grails applications.

So, take them for a spin and improve your Grails applications!

Grails testing tips

Don't get me wrong, Grails is fantastic, but the documentation sometimes is lacking or completely wrong. I realize that development is moving so fast with Grails that documentation gets overlooked but this can be a source of frustration.

Often my frustration comes when I am trying to write effective tests and the documentation often send me (and my co-workers) down the wrong path.

In an effort to keep others from making the same mistakes I have made in the past, you may want to just check out the MockUtils implementation to see what Grails really has to offer during testing.

If you aren't familiar with testing in Grails then check out the sample chapter (which happens to be all about testing) from the book Grails in Action. If you don't already have a copy of the book, I recommend you get one. It's fantastic!

P.S. As a bonus here are some nice examples of testing REST controllers.

Got Groovy?

One of the things I really love about the Groovy language is how quickly you can accomplish things. There has been a lot of thought put into making the language easier to use and it really shows.

All though I highly recommend reading through the GDK to really understand all the great stuff that Groovy has to offer, sometimes you just need to know how to do something quickly.

I stumbled upon a nicely organized Groovy version of the PLEAC (programming language examples alike cookbook). It's chalk full of examples on how to quickly accomplish common tasks and is a great demonstration of the power of Groovy.

Grails custom constraints

Geoff Lane over at Zorched.net has a great post about creating your own custom constraints for validation in Grails. While I didn't take the same exact route as he outlined (I didn't create a plug-in) it was amazing how quickly and easily you can add custom constraints to Grails.

I also happened to notice that AbstractConstraint allows for some additional options, such as vetoing which could come in handy if you want to take a "fail fast" approach to constraint violations.

So head over there, and read up!