Tuesday, September 14, 2010

Pulling the configuration for the mail plugin for grails from the database

I have a grails application that sometimes needs to email users. I am using the mail plugin, but that expects the configuration (SMTP server, etc) configuration to be in the application's Config.groovy file. I wanted to make it possible for the site administrator to change the configuration. This way, they could create a specific email account with Google just to send email for the application.

After some research, I discovered that the configuration could be altered at runtime by getting the mailSender injected into my code, and setting its properties as needed.

Next, I needed a domain object to represent my configuration. I tried using the @Singleton annotation, but that does not play well with domain objects. I ended up writing getInstance() myself:


static long instanceId = 1l;
static EmailServerConfig instance;

static synchronized getInstance() {
if (!instance) {
instance = EmailServerConfig.get(instanceId);
if (!instance) {
instance = new EmailServerConfig()
instance.id = instanceId
instance.save()
}
}
return instance;
}
This can only work with
static mapping = {
id generator:'assigned'
}


and even with that, I was unable pass the id parameter to the constructor, that's why I set it separately after the EmailServerConfig object is created.

Then all I need is to define the GORM event handlers afterLoad, afterInsert and afterUpdate to apply the values from the database to the mailSender.

Finally, I made sure to encrypt the SMTP authentication password on its way into the database, and to decrypt it when retreiving it. Thanks to Geoff Lane for this blog post on how to handle that with codecs:
class EmailServerConfig {
String password
String passwordEncoded
//...
def afterLoad = {
password = passwordEncoded?.decodeSecure()
updateMailSender()
}
def beforeUpdate = {
passwordEncoded = password?.encodeAsSecure()
}
//...
}

Then all I had to do was write the controller to let site admins edit those values. The controller uses EmailServerConfig.getInstance(), since we're simulating a singleton.

Things to remember:
  • onLoad is called before the values are loaded into the object, so the values are null; use afterLoad instead
  • beforeInsert, beforeUpdate and beforeDelete are called only once the object is validated, so the constraints have to allow a null value for passwordEncoded
  • Grails tries to instantiate domain classes at startup, and @Singleton prevents that, that's why you can't have @Singleton on a domain class
  • If you don't create an EmailServerConfig in the Bootstrap, and you do not provide any configuration in Config.groovy, the mailSender will default to trying to send mail through localhost:25. This may work, depending on your setup.

Saturday, July 3, 2010

RoCoCo 2010 in Montreal - Recap and Impressions

The Recent Changes Camp (also called RoCoCo) was held last weekend (June 25-17) in Montreal. I'm not sure how I heard about it. Possibly someone I follow on twitter commented on tikiwiki, and I checked it out and saw that they were sponsoring a conference near me. I do not "wiki". We use one at work, but I have felt no drive to get involved in the open ones such as wikipedia or wikiHow. But I decided not too long ago to be move involved in what's happening near me, and to actually meet people face to face once in a while. So I went. Their website made it look open enough.

It turns out, that was a good idea.

The first few people I met were welcoming and friendly, so I stayed. "I don't really have anything to do with wikis" was met with "that's cool", and "whoever ends up being there is whoever was meant to be there". I learned first-hand about the openspace technology (well, I'd call it more a philosophy or concept, but it seems that it's usually referred to as a technology). When we got together to set the agenda, I expected people to act like I've always seen people act, shy: "you go first" "no you do". But they couldn't wait for the presenter to give them the floor to introduce the sessions they wanted to host. I suppose I shouldn't have been surprised; many are active wiki or open software contributors, so they do not wait for others to do what they want done - they are doers. Still, for me that set the tone do the day - a very proactive sort of tone. Which eventually led me to co-host a talk (at Mark Dillon's invitation). I hadn't planned on coming back on the Saturday, but I was learning, I was talking to people, I felt part of the group, so I came back.

Why wiki, why not wiki, and why don't more people contribute?

The first session I attended was led by Mark Dillon, and asked "Why wiki?". Many issues were discussed, as you can see from the notes. We discussed how things are, how people use wikis, and how it can make things better. We also looked at some reasons why people do not wiki. The technical aspect came up (wiki syntax can be cumbersome), but I felt they were leaving the "people aspect" out of it a bit- being shy, or... something. We did cover a lot of the features and how they can act to invite users to contribute, or add to the "conversation". After the talk, Mark asked if I would co-host another one on barriers to entry. I don't know if it was his strategy to gently push a newcomer, of it was just the action of a natural leader - either way, it worked, and I accepted. The session happened the next day. I think we covered a lot of "personal" aspects. I personally feel that while the technical complexity may keep some people away, many just don't want to write, either because they are shy, or do not feel the drive to do so. I wanted to explore these in more details, and we certainly did. What came out of it for me, why I think many of us don't contribute, is that we feel we need the permission to edit someone else's text. Even when we know how the system works, we may want to make sure we're not going to hurt the other person's feelings, or possibly miss an important point they were making, or an angle they were trying to give. Another major obstacle is the lack of feedback. If we don't believe that others will review our work and improve on it, there is little to motivate us. What I didn't know is how much of a community the contributors end up creating, and that it is an important factor to current contributors. A very interesting and enlightening set of discussions.

Since then, I have read an article (thanks to Seb Paquet for pointing it out) showing some empirical measurement of the benefits of methods used to encourage people to contribute. I have also encountered the concept of "diffusion of responsibility" and "the bystander effect". The feeling that we don't have to do something; someone else will. I was somewhat familiar with the concept, but now it has a name. And I'm sure it plays a role in whether one contributes to wikis, as well as pretty much any open venture.

Structured wikis, semantic wikis, Semantic web

Here, I mostly took notes, trying furiously to make sense of all the information provided. Structured wikis are those where contributors are asked to fill in fields, rather than (or as well as) come up with the text of the article. It may feel more like entering data into a database. For example, for an entry on an author, the user may be asked for a date of birth, a date of death, schools attended, genre, and a list of titles.

A semantic wiki is one where the occurrence of information with a particular meaning ("this is a city", "this is a person", "this is a date") is annotated as such. This then allows the software to make links between other things that also have that information, such as other authors who were born in the same city, or events that occurred on the same date.

The power of both these wiki styles is greatly improved when they are combined - structured semantic wikis create a wealth of information that can be interrelated by a machine to create a rich set of information.

This is also where I learned about the existence RDF, and the RDF triple: a relation is expressed as 3 components: the subject, the predicate, and the object. I was taken back to 6th grade grammar, but of course it makes sense. If you want to describe a relation you need the the origin of the relation, the type of relation, and the target of the relation: "Bob knows Peter", "Steven King wrote The Stand". I believe someone brought up microformats, which can encode the same information, but differently, in a way that's more accessible on the web. There is so much here to try to make sense of, I'll be spending some serious time puzzling it out. But I think the potential there is obvious. Calling it "Web 3.0" may be an exaggeration, but I don't think it's far off.

Open companies and reputation systems

Bayle Shanks presented the concept of open companies where employees are those who want to work on the company's project, and they get paid according to a rating system by their peers. When I first sat in that session, I expected it to be about open companies in the sense of information flow, not a true open source parallel. Before the talk, I would have thought the whole concept simply crazy, but the culture of open spaces (or this particular one?) is such that we all listened, commented, suggested, brought up issues, without anyone being offensive or dismissive. I'm still not sure it can work in a general setting, but I applaud Bayle for wanting to give it a shot, and I can see some applications where it has some definite potential.

The reputation system he wants to use is quite intriguing. The system sounds simple enough, but I'm not following all the repercussions of money distribution scheme. The interesting bits is that the people who contribute the most also end up being the ones who have the bigger influence on the distribution of rewards (money, reputation, gold stars). This can be useful in a number of non-money applications, too, as a way to recognize the contribution of volunteers, for example. Maybe all wikis should have that!

And here is why keeping an open mind pays. If I had chosen not to listen to the open company part of the session I would never have heard about the reputation system. And boy am I glad I did!

Multilingual wikis

The resources on the net are still mostly in English, but the web as a whole is getting a lot better. At least now, a number of HTTP requests include an Accept-Language header that makes sense for the context. Still, users do not always want to browse a particular site in the language configured in their browsers.

The question was, "how do you handle multilingual wikis"? This can be particularly hard to do when original content can come from any of the supported languages. If someone updates the French version, how do you make sure the changes are included in the English version? Wiki translation shows some promise, but I think there are still more questions than answers on that topic, both for wikis and for websites and social networks in general. I still haven't figured out how to Facebook or Twitter bilingually.

Intent Map

In another session, Seb Paquet introduced the idea of a way to find people who want to work on the same sort of things you want to work on. We discussed this, and by the end of the session, we had agreed to turn this into a project. Others in the group were already familiar with RDF (see section above on semantic web). Someone else brought up FOAF, a specification for identifying people and relations between people. Someone else brought up DOAP. Microformats came up again. I try write down everything I'm going to have to learn about. And yet, I volunteered to code it all. Code what? well, that's the question! I'm not sure what I got myself into. During the session, I mentioned that one way to get widespread visibility would be with a Facebook application, so I have started playing with writing a Facebook application. Since I don't want to worry about hosting the application (and the scraper, and the database that will be needed to support it all) I also started learning about Google App Engine. I even wrote a very simple application that displays inside Facebook, but I'm still sorting out authentication and permissions. It is quite a challenge, but I need to get out of my comfort zone, so this is perfect. I get to play with new (to me) technologies, learn new specifications, and tons of new concepts. I hope I don't disappoint my teammates. I better get coding.

What now?

I met tons of new people. I hosted a session. I am now an official contributor to the RoCoCo2010.org wiki (I wrote up some session notes). I volunteered to implement something around a spec that doesn't even exist, with tools and APIs I don't know. This is not the same-old, same-old, but I'm okay with that. And I'm going to do it again, when the opportunity arises again. Thanks to everyone who was there for making it so energizing.

Sunday, June 27, 2010

Unordered notes from the RoCoCo unconference in Montreal this weekend

I decided to attend the Recent Changes Camp 2010: Montreal
Here are a few notes I made for myself, and that I'm sharing now, at least until I get around to writing a proper entry:

Wikis (well, wiki software) could be a way to implement addventure (a "you are the hero" collaborative story telling game originally written by Allen Firstenberg). Wiki red links are very much like "this episode has not been written yet".

Wikis synthesize (focus), where forums divide (or disperse in their focus).

Ontology vs folksonomy.

Look into:
HTLit
Inform 7
Universal Edit Button
Etherpad
Semantic web; the semantic triple: [subject, predicate, object]
Resource Definition Framework (RDF), SPARQL (query language for RDF)
confoo
appropedia
Google wheel
microformats


To read:
The wisdom of crowds


I also got to lead a session (which is a lot easier than you might expect since all the participants are interested in the topic anyway - or they leave, and because they participate willingly). And we started a new project!

Thursday, June 10, 2010

How to transform one type of object to another in Groovy/Grails (as long as it's not a domain object)

I've been working on a system that will be using remote calls to communicate between a client (browser, mobile phone, possitbly a GWT client) and the server. The client sends a request, and a grails controller returns a grails domain object encoded using JSON. Relatively straight-forward stuff, but I hit a few snags. I was thankful when I discovered http://blog.lourish.com/ which goes into some details into how to make it happen. Detailed post are here, here, and here.

I debated using the ObjectMarshaller to restrict the data sent (afterall, the client doesn't need to know the class name of my objects), but in the end, I decided to use Data Transfer Objects. I can see a future development where these objects will be used as commands, for example.

The problem that's been keeping me awake tonight, tho, is in the translation from domain object to DTO. Based on my reading, it looked like I could transform any kind of object into any other kind of object, as long as the initial object knew what to do.
class User {
// grails will contribute fields for id and version
String lastName
String firstName
Address workAddress //
Address homeAddress // The client does not need that info and SHOULD NOT ever see it
static hasMany [roles: Role, groups: Groups]  // etc

doThis() {
//..
}

doThat() {
//...
}
}

class UserDTO {
String lastName
String firstName
}
How do you take a User object and make a UserDTO out of it? Well, you should certainly have a look at Peter Ledbrook's DTO plugin. But for my needs, I thought I'd stick with something simpler. Just use the groovy "as" operator.
All you need to do something like
def dto = User as DTO
is to have User implement asType(Class clazz) and to handle (by hand) the case where clazz is DTO:
class User {
// same fields as before, etc
Object asType(Class clazz) {
if (clazz.isAssignableFrom(UserDTO)) {
return new UserDTO(lastName: lastName, firstName:firstName)
} else {
return super.asType(clazz)
}
}
}
All works well. Unit tests confirm, there's nothing to it.
void testUserAsUserDTO() {
String lastName = 'Lovelace'
String firstName = 'Ada'
User u = new User(lastName: lastName, firstName: firstName);
UserDTO dto = u as UserDTO;
assertEquals(UserDTO.class, dto.class)
assertEquals(lastName, dto.lastName);
assertEquals(firstName, dto.firstName);
}
Integration test. I want to make sure my controller sends the right data
The controller:
def whoAmI = {
def me = authenticateService.userDomain() // acegi plugin; this returns a User
if (me) {
def dto = me as UserDTO
render dto as JSON
} else {
render [error: "You are not logged in"] as JSON
}
}
The test:
class RpcWhoAmITest extends ControllerUnitTestCase {
void testWhoAmI() {
String lastName = 'Lovelace';
String firstName = 'Ada';
User u = new User(lastName: lastName, firstName: firstName)
mockDomain(User.class,[u])
mockLoginAs(u)
controller.whoAmI()
def returnedUser = JSON.parse(controller.response.contentAsString)
assertNotNull(returnedUser)
assertEquals(lastName, returnedUser.lastName)
assertEquals(firstName, returnedUser.firstName)
}
}
And that... fails! The message is
org.codehaus.groovy.runtime.typehandling.GroovyCastException: Cannot cast object 'my.package.User : 1' with class 'my.package.User' to class 'my.package.rpc.UserDTO'
at org.codehaus.groovy.runtime.typehandling.DefaultTypeTransformation.castToType(DefaultTypeTransformation.java:348)
at ...

What went wrong? The call to mock my domain object is what went wrong. It replaces my asType(Class clazz) with its own. Fortunately, that's relatively easy to fix. I needed to override the method addConverters in grails.test.GrailsUnitTestCase to replace asType(Class) only if it didn't already exist (in my test class):
@Override
protected void addConverters(Class clazz) {
registerMetaClass(clazz)
if (!clazz.metaClass.asType) {
clazz.metaClass.asType = {Class asClass ->
if (ConverterUtil.isConverterClass(asClass)) {
return ConverterUtil.createConverter(asClass, delegate, applicationContext)
}
else {
return ConverterUtil.invokeOriginalAsTypeMethod(delegate, asClass)
}
}
}
}


Sadly, after all this work, I deploy, launch, and still get GroovyCastExceptions. It turns out that the instrumentation of domain class objects essentially throws out my "asType()" method. In the end, I switched to the DTO plugin (which post-instruments the domain object to do it's own stuff, something I considered doing, but at some point, the "quick, home-made solution" just isn't.

Monday, May 31, 2010

WebCom Part 3

The third speaker in the rapid-fire keynotes was Adele McAlear with Death and Digital Legacy.

That presentation was definitely nowhere near as much fun as any of the other ones, simply due top the morbid topic. It is a brand new domain - what do you with all the electronic assets when their owner (producer, creator) dies? Who has the rights to decide what happens, and how are the service providers handling it?

First, Adele showed us the breadth of our "online footprint". From email accounts to flickr uploads, blogs, tweets, WOW characters... Each service provider may have different policies for dealing with a deceased person's account, but in reality, only Facebook has a stated policy. And what about paid subscription services? When a person dies, the credit card they used to maintain that service is cancelled, and unless a survivor has access to the account, they can't change the credit card on file (I wonder what happens in the cases where the service have "gift subscriptions", such as "buy Person a pro account"). The survivor can't gain access to the account because it is tied to an email account, to which, in all likelihood, they do not have access.

So what solutions do we have, if we want to leave something behind, if not for ourselves, for our friends, fans, followers?

First, we should make a list of our digital assets. What accounts we have, and how we'd like them to be preserved after death. For example, we may want blogs to be preserved or archived, but we may think that our twitter account history is just not worth the effort. We should make sure that our family knows about our online accounts, too, so they know what to expect.

Then we appoint a digital executor. This is someone with whom we will have discussed the matter, and who will be responsible for our digital legacy. This does not have to be same person who will execute our will. Then, we create an email account exclusively for this purpose, and set it up as a "backup email account" for our regular email account. This way, when the executor wants to take over our main email account, they only have to request a password reset be sent to the backup email account. The password to the backup email account should be kept with our will. From there, the executor will be able to gain access to other accounts by requesting a password reset.

Saturday, May 29, 2010

WebCom Part 2

Sean Power, Communilytics

Communilytics, is the analysis of how specific information flow through online communities. In his presentation, Sean encouraged us to look a the numbers at a deeper level than simple page-views or the number of followers. But before we can mount a successful campaign, we have to decide what we want the accomplish with that campain: make more money, gain attention/recognition, or improve our reputation. How deeply we want to get involved in the community we're targeting (whether we want to search, join, moderate or run it) will affect which tools and technologies to use. There are 8 social platforms(group/mailing list, forum, real-time chat, social network, blog, wiki, ), with different dynamics, and each can be addressed at different levels of implication. The metrics, then, will depend on the tools used. Each business is different, we know best what's right for our business.

He also brought up the AARRR model by Dave McClure: Acquisition, Activation, Retention, Referral, Revenue.

He then gave some examples of how information flows through communities. When one person posts a tweet, their followers see it, But if it's a tweet of interest, the recipients may want to let their followers know about it, and re-tweet it. The reach of a message, then is the followers, those who receive the message as a retweet, and those who receive the retweet retweeted, etc. But some people may cross cross social platform - they may put a tweet on Facebook, or send it through email.

This presentation is where I also found out the format definition of "going viral". That's when the average number a person tells the message to is greater than one. (So on average, everyone who gets this message will forward it).

To get a wide reach, sometimes the best way is to find a few seeds who, because of their respectability and following, will insure a wide, receptive audience for the message. The sites Twinfluence, tweetreach and TwitterAnalyer can help find out the reach of tweeter user and messages.


Tuesday, May 25, 2010

WebCom Montreal

I was lucky enough to find out about this event a few days before the fact. I only participated in the free portion: 5 presentations of 15-20 minutes about different aspects of Web 2.0 and social networks.

Here are some notes and thoughts about the presentations:

Chris Heuer, Serve the Market

Originally, organisations served their community: the blacksmith, milkman, general store were all there to serve the needs of the community. The market has now shifted to products. Many still provide a service, but this isn't how we view the market anymore; but we should. He pointed out that how we chose to interact with organisations is through "filters" that become thiner with trust; the more flow, information flow, the deeper the relation, and the more likely people will buy.

Serving the market is leadership, not management

and the parting quesiton,

is profit the only purpose of business? or are we able to transcend our current thinking?


This was a great start. I felt like screaming "Yes!! that's right!!" on many occasions. I think we most certainly can work in a market focusing first on the service we want to render, and that money is not the goal in itself. Of course, we wont be able to provide this service if we don't make enough money, so of course money is important, but all too often, we use money as a measure of our success, and that's a false indicator. The positive impact we have on our community can be in the form of the product or service rendered, but it can also be in terms of providing employment, and spreading a little bit of happiness to clients, employees, and to every member of the community.

Next: Sean Power, Applied Communilytics In a Nutshell

Thursday, May 20, 2010

Motivation

For most software developers, programming is more than just a way to earn money. What puts a smile on our faces as we think about upcoming tasks? What makes us think about the current problem when we wait in line, while we're driving, or as we fall asleep? Sure, we do it because it has to be done, but often there are much more personal reasons. Where does the satisfaction come from? We are all different, and different things make us tic, but all the cases fit in 3 categories: personal, interpersonal, and global. The fun is in figure out who we are based on what's important to us

Personal

  • Feeling smart: This is a very good feeling, whether it's because we've beat the machine into submission, or because we've found a whole new way to use an old tool. It doesn't mean outsmarting another person, but lining up ideas in a productive way
  • Aesthetics: Coming up with a beautiful, elegant design. Simplifying something complicated. Making all the parts fit nicely
  • Achievement: Going beyond our abilities, taking it one step further
  • Learning: New tools, new ways of thinking. We prefer learning things that change how we thinking about the problem. The more ways we can shape our minds around an issue, the more likely we can come up with an elegant solution; or a solution at all.
  • The big "DONE" stamp: when we get to say something is done, out the door, complete. Sometimes we have to compromise - we'd like to tweak this a bit more, or refactor that, but like an artist, talent isn't only knowing how a piece can be made better, but also knowing when to stop.
  • Glory: well, okay, that's pushing it a bit, but the recognition of others, whether our peers, customers, or the whole wide world is a very powerful motivation.
Interpersonal

We are often seen as solitary workers, but in truth, our job cannot be done totally alone. We value good relations with our coworkers, mentors, employer, and clients.

  • Coworkers: we often need to rely on others, whether to show us what they've done, to discuss an idea, or do share the workload. Good relations with them come from listening to ideas, asking question, and pointing out issues in a respectful manner. It works even better when we can be friends with our coworkers, and spend time together outside the work environment, but that's not necessary.
  • Relations with the employer can be very productive when we know what's expected of us, what the boundaries are, and when we know we can meet our targets. We much prefer having some freedom in how we attain these objectives, and when we have input in setting the targets. When employees are asked to set the target themselves, they tend to shoot for higher accomplishment, and they are more likely to reach them. This is true not only for software developers, but in pretty much any field.
  • Clients (in a consulting or customization setting) are also part of the interpersonal aspect of a developer's life. Some of us hate talking to the client, but for others, knowing the person who will use the product, knowing how they will use it and why, can help us propose solutions they may not have thought of. It may require us to think differently, to use a different language so we can truly connect with the client in terms they understand, but that just keeps our minds nimble.
Global

For some of us, the greater good is what drives our actions. We can make a difference by the work we do, whether it's through software that favors sustainability, the people we're helping, teaching/education we support, the time our software will save thousands of people, the list goes on. We all have causes we support, some more ardently than others, and when our work allows us to promote them, or help them along, we derive even more satisfaction from our efforts. We build a legacy, even if it's all too often anonymously.

Clearly, money is not the only reason we develop software. If it were, there would be no Open Source movement.

We all feel the pull of these motivations differently. For some, doing good for goodness's sake is plenty; others want recognition. I'm generally motivated mostly by the personal and interpersonal aspects of the work. I value the recognition of my peers more than that of the population at large.

What motivates you?