Checkout EasyPattern for Javascript and NodeJs!
During some work on our internal tool, I have created EasyPattern, which is a readable alternative to regular expressions.
It is great to match urls with ease, and keep the code super readable and configurable.
For example:
Basic testings
var easyPattern = require("easyPattern");
var pattern = easyPattern("{file}.js");
pattern.test("archive.zip"); // false
pattern.test("index.js"); // true
Basic matching
var pattern = easyPattern("{folder}/{filename}.js");
var result = pattern.match("foo/bar.js");
//result = {folder: "foo", filename: "bar"}
Wildcard matching
var pattern = easyPattern("*.{extension}");
var result = pattern.match("/root/folder/file.exe");
//result = {extension:"exe"}
Advance matching
var pattern = easyPattern("{*}/{filename}?{*}");
var result = pattern.match("www.site.com/home/hello.js?p=1");
//result = {1:"www.site.com/home", 2:"p=1", filename:"hello.js"}
You can download and play with it here
Problem:
We had an integration test which creates a spring ClassPathXmlApplicationContext and while doing so the test blew up with a NoSuchMethodError. It turned out that we had conflicting versions of dependencies on Spring artifacts. This in itself is not an unusual problem – such problems are addressed with the maven dependency plugin using the verbose option. However, what do you do when the maven plugin is wrong?
Investigation:
We started to dig in and we found that the AbstractAutowireCapableBeanFactory in its getTypeForFactoryMethod method tries to access the GenericTypeResolver resolveReturnTypeForGeneric method and fails on a java.lang.NoSuchMethodError: org.springframework.core.GenericTypeResolver.resolveReturnTypeForGenericMethod(Ljava/lang/reflect/Method;.
Initial investigation and googling we found that the method was added in 3.2.0 while we’re supposed to be running with 3.1.1.
Further investigation determined that spring-data-mongodb depends on spring framework in range [3.0.7-4) 1 and because maven takes the latest available version given that range 2 it tried to take 3.2.2.
Note that the above changes a bit given a conflict between an explicit version dependency and a range dependency but, IINM, when determining the dependencies of spring mongo there is no conflict.
The problem was further masked by two symptoms:
- We have other projects that use this pattern and have no problem- This was explained by the fact that the conflict-resolving mechanism of maven chooses the nearest version it finds by default 3 and since all other projects which need spring-data-mongodb depend on this project they were lucky enough to grab the 3.1.1 version and not 3.2.2
- dependency:tree shows it brings 3.1.1 while bringing 3.2.2- Since the stack trace showed other results I wrote a test which checks from which jar each of the above classes comes from and verified that indeed the AbstractAutowireCapableBeanFactory class arrives from spring-beans 3.2.2 and not 3.1.1 as "mvn dependency:tree" showed (a big thanks to http://bit.ly/10zD1iV for the code snippet of finding the jar of a class in runtime).
Maven dependency:tree output showing spring-beans:3.1.1 is used in the artifact
Test which proves spring-beans:3.2.2 is used in the artifact (asserting what the jvm in the error was saying)
The reason spring-core artifact came in 3.1.1 when spring-beans came as 3.2.2 is that our framework explicitly depends on spring-core and this artifact explicitly depends on the framework. This means spring-core 3.1.1 from framework is 2 hops which is shorter than the 3.2.2 from spring-data-mongodb.
Solution:
Depend on spring-data-mongodb while excluding spring-beans like so:
The Open question mark:
Why dependency:tree (in verbose mode) did not show that it brings spring-beans in 3.2.2 but in 3.1.1 while explicitly specifying that spring-core 3.2.2 was removed due to conflict? I chalk this up to a bug in the dependency plugin.
Update:
Thanks to @Samuel_Langlois we confirmed that the “open question mark” is indeed a Maven Dependency Plugin bug which was resolved in 2.5 and so using 2.7 and forward will show you the correct info.
- http://repo1.maven.org/maven2/org/springframework/data/spring-data-mongodb-parent/1.0.1.RELEASE/spring-data-mongodb-parent-1.0.1.RELEASE.pom ↩
- http://www.maestrodev.com/better-builds-with-maven/creating-applications-with-maven/resolving-dependency-conflicts-and-using-version-ranges/ ↩
- http://www.maestrodev.com/better-builds-with-maven/creating-applications-with-maven/resolving-dependency-conflicts-and-using-version-ranges/ ↩
So we have some Scala code that consumes text from an InputStream from an HTTP response. So, like any good Scala developer, I handed over the response to a function that returns a String. This function performs some validations such as checking that the response status is 200, then consumes the InputStream. This was done by creating a scala.io.Source from the InputStream, then calling source.mkString to consume the response body.
Or so I thought.
Apparently, scala.io.Source is an Iterator[Char] and inherits the mkString() function from TraversableOnce. Calling a TraversableOnce.mkString() appends all members of the TraversableOnce instance to a StringBuilder – which is fine, unless this instance is actually an abstraction over an IO-bound stream, whereupon it consumes the stream byte-by-byte. This is, as some of you might now, a terribly inefficient way to consume an InputStream, especially a network-bound one. When running a thread profiler, we were horrified to discover that we are spending almost 70% of the time waiting on IO consuming these HTTP responses.
The solution?
Source.fromInputStream(response.getInputStream, "UTF-8").getLines.mkString
Test code I used to prove the problem:
package com.wixpress.scala
import java.io.ByteArrayInputStream
import scala.io.Source
object SourceBenchmark {
def main(args: Array[String]) {
val bytes = new Array[Byte](1024 * 1024)
val is = new ByteArrayInputStream(bytes)
val mkStringTime = measureInNanos {
Source.fromInputStream(is).mkString
}
val getLinesTime = measureInNanos {
Source.fromInputStream(is).getLines().mkString
}
println("Time with mkString: %s, time with getLines: %s".format(mkStringTime, getLinesTime))
}
def measureInNanos(f: => Unit) = {
val before = System.nanoTime()
f
System.nanoTime() - before
}
}
This output (from my late 2011 MacBook Pro):
Time with mkString: 548707000, time with getLines: 1868000
This is a difference of about x600 in favor of the version with getLines()!
Edit: I have created an issue in the Scala issue tracker for this bug.
We’re proud to announce underscore, the Israeli Scala user group, with a meetup.com group to follow soon. Join us in building the local Scala community and stay put for exciting announcements in the near future!
Just wanted to share a nice piece of code I created..
During my recent experience with NodeJs, there was one thing that bugged me the most – in Node, to include another module into the scope, you use ‘require()‘ to import the other module.
The thing is, that if it is not a third party module, you have to specify the full path in your project (absolute or relative) in order to do so. it usually looks something like this:
require('../../../commands/MyCommand.js');
Not only it is ugly, it is terribly hard to refactore the code afterwards (IntelliJ’s refactoring fails to do it).
—-
So, I created this module called rekuire.
rekuire scans the project paths upon first execution, so all you need to specify, is the file name!
no paths! (it saves a bunchload of time)
rekuire('MyCommand.js');
In order install, using npm, type “npm install rekuire“!
And don’t forget, you are more then welcome to check it out and contribute to it on github.
This week, I was handed a nice task – to integrate with some external service we use, that has no API. All communication with the service was supposed to be mimicking a multiple step scenario of a user integrating with some forms. This means, cookies, sessions, and all the wonders of HTTP protocol..
And, just to add giggles, the whole component is to be written in and act as a part of a NodeJS server.
So, I thought to myself, first thing first; I configured a new NodeJs project with a Jasmine test environment, and started writing my integration tests.
But unfortunately, not only my remote service had no API, there was no way I could create an embedded local service for me to test against!
Popped up Fiddler (or Charles, whatever..) and sniffed the HTTPS traffic as I filled in the forms (Fiddler does it quite elegantly), and created a spec, specifying how should each and every packed look like.
To test it, I created a little HTTP service module, that can store its last request and even stub a response behavior.
It supports HTTP and HTTPS, and made my life easy
Here is some usage examples:
Setting it up:
beforeEach(function(){
server = new TestableServer(3030);
server.respondWithStatus(302);
server.start();
});
afterEach(function(){
server.stop();
});
And making the actual test:
describe("when executing",function(){
it("should create the right request to the service",function(){
cmd.execute( buildTestRequestData(email,session) ,function(){});
waitsFor(function() {
return server.lastRequest != null;
}, "a request to arrive" , 1000);
runs(function(){
expect(server.lastRequest.body["email"]).toBe(email);
expect(server.lastRequest.headers[SESSION_COOKIE_NAME]).toBe(session);
});
});
});
You are more then welcome to grab the source code here:
TestableServer.js
Launch a Web App in 3 days to over 30 million Wix users!
Join the first ever open Wix TLV Hackathon, March 7th to 9th, at Wix HQ in the Tel Aviv Port
Rock star developers, Internet geeks, hungry entrepreneurs, ninja designers, marketing gurus, master hackers, striving product managers – you’re all invited!
Join the Wix TLV Hackathon where you can learn, network, innovate and create a killer web app that will reach millions of Wix users. Get free office space and mentorship to turn your vision into reality.
Wix developers, designers, product managers and UX experts will all be there to help as you put your genius to work.
Up for the challenge?
Schedule:
Thursday, March 7 – Pre Hackathon Workshop
18:00 Meet up and drinks – Scope out the competition with liquid courage!
19:00 David Schwartz – Wix App Market: Designing a Winning Web App
19:30 Yoav Abrahami – Working with Wix SDK
20:00 Entrepreneurs’ & innovators’ 1-minute pitches!
21:00 Drinks and mingling – Shape your idea, build your team, and plan what you’ll do with all that Sweet Exit Cash!
Friday, March 8 – first day
09:00 Get together and breakfast – carb load for the day ahead
10:00 Choose your workshop:
Yoav Avrahami- Developing Wix Apps using Google App engine
Ran Mizrahi- Developing Wix Apps using PHP
13:00 Lunch and team-up with your future co-founders
19:00 Dinner to wind down (or up, if you’re planning on a White Night)
Saturday, March 9 – second day
09:00 Breakfast with coffee, lots and lots of coffee
13:00 Lunch with more coffee
17:00 Last change to submit a killer App that will take the web by storm!
17:30 Demo Presentations – it’s money time!
19:30 Announcing the Wix TLV Hackathon Winners
20:00 Eat, drink and party
About Wix & Wix App Market
Since 2008 more than 28 million websites have been built using Wix’s state of the art, drag-and-drop web building editor. Renowned for its creative, no-coding-needed flexibility, small businesses and individuals have flocked to the Wix platform. More than 4 million people engage with the Wix editor each month, publishing more than 1 million new websites during that same time frame.
With this as a foundation, a massive new web developer opportunity presented itself in October of 2012 as the Wix App Market launched and was integrated within the Wix editor. How big an opportunity? At present there are more than 500,000 App installations through the Wix App Market every month. Currently, with less than 70 Apps in the App Market, the average App is getting more than 7,000 installations per month.
The Wix Hackathon Website:
In this post i will demonstrate a nice trick to add “extension methods” for a given class.
Note: The feature is based on implicit class feature and available only in Scala 2.10
case class Cat(name : String, kittens : Seq[Cat] = Nil)
object implicits{
implicit class SmellyCat(cat : Cat) {
def hasKittens : Boolean = cat.kittens.size > 0
}
A class Cat has a name and a list of kittens. in this example we add a method named “hasKittens()” to the Cat class. This is done by creating the implicit class SmellyCat which implicitly wraps our Cat instance when calling “hasKittens()”.
The following test shows the usage of this feature:
class ExtensionFunction extends WordSpec with ShouldMatchers with MustMatchers{
"extension function " should {
"extend function of cat " in {
import implicits._
val cat1 = Cat("Philip")
cat1.hasKittens should equal(false)
val cat2 = Cat("Kozmo", Seq(Cat("Mitzi")))
cat2.hasKittens should equal(true)
}
}
}
To use the extended function you need to import the class (import implicits._).
Note: implicit class cannot be on the top level of the project and must be wrapped inside an object
In this post i will cover usage of the Slick library, which provides a Scala-centric API over JDBC.
Slick offers 3 flavors for querying the DB – I will focus on the plain SQL flavor.
Note: Slick is only compatible with Scala 2.10
Maven dependency:
<dependency> <groupId>com.typesafe.slick</groupId> <artifactId>slick_2.10</artifactId> <version>1.0.0</version> </dependency>
DB table
CREATE TABLE animals ( id varchar(50) NOT NULL, name varchar(256) NOT NULL, date_created_nano bigint(20) not NULL, PRIMARY KEY (id) );
We will represent the Animal domain object using the following case class:
case class Animal(id: UUID, name: String, dateTime: DateTime)
Note: that the first parameter is complex type (UUID) and the 3rd parameter is also Complex type (Joda DateTime)
We will hide all DB operations using a DAO as described by the following Trait:
</span>
<pre>trait AnimalDao {
def allAnimals: Seq[Animal]
def animalById(id: String): Option[Animal]
def deleteAnimalById(id: String): Int
}
And the implementation:
import scala.slick.session.Database
import Database.threadLocalSession
import slick.jdbc.{StaticQuery => Q, GetResult}
import javax.sql.DataSource
import Q.interpolation
import org.joda.time.DateTime
import java.util.UUID
class DefaultAnimalDao(dataSource: DataSource) extends AnimalDao {
lazy val db = Database.forDataSource(dataSource)
def allAnimals: Seq[Animal] = {
db.withSession {
Q.queryNA[Animal]("select * from animals").list
}
}
def animalById(id: String): Option[Animal] = {
db.withSession(
sql"select * from animals where id = $id".as[Animal].firstOption
)
}
def deleteAnimalById(id: String): Int = {
db.withSession(
sqlu"delete from animals where id = $id".first
)
}
implicit val getAnimalsResult = GetResult(r => Animal(UUID.fromString(r.nextString), r.nextString, new DateTime(r.nextLong())))
}
- The DAO is dependent on a javax.sql.DataSource
- In the second line we create a Slick Database object using the provided DataSource
- allAnimals() – uses the queryNA[Animal] function and returns a Seq of Animals.
- animalById() – uses the sql String interpolator. The .as[Animal] method tells the function to return an instance of Animal and .firstOption() returns only the first result, wrapped in an Option
- deleteAnimalById() – uses the sqlu String interpolator for update/insert – the result is always an Int representing the number of affected trows
- the implicit val getAnimalResult is very important. It tells Slick how to create the object and encapsulates the logic constructing the complex types mentioned earlier .
How many times did you find yourself looking at an API which accepts some sort of string or int-based identifier and not being sure what the identifier should represent? This sort of thing happens a lot when you use several different DAO or Repository objects. For instance, I may have:
case class Cat(id: String)
case class Dog(id: String)
trait CatRepository {
def get(id: String): Cat
}
trait DogRepository {
def get(id: String): Dog
}
Note how easy it would be to accidentally pass a dog id into the get method of CatRepository.
Now, back in the happy days of Ada and Delphi (and Pascal, of course), you could define your own types; for instance, I could write:
type CatId is String type DogId is String
for which the compiler will produce an error if I try to pass a DogId instead of a CatId. Another highlight of using specific types for identifiers is that we now provide a higher-level abstraction over the concrete type of the identifier. I no longer care that a DogId happens to be a string – I just treat it as a DogId everywhere in the system.
Scala also has the type keyword, allowing us to define custom types. However, this only defines a type-alias, not a truly separate type in the type system. As a result, I could pass a string instead of DogId or even worse, a CatId instead of DogId:
type CatId = String
type DogId = String
case class Cat(id: CatId)
case class Dog(id: DogId)
trait CatRepository {
def get(id: CatId): Cat
}
trait DogRepository {
def get(id: DogId): Dog
}
...
val id: DogId = "fido"
val cat = catRepo.get(id) // this compiles because DogId is really a String
So, we need to take a more drastic measure to cause compilation to break if I try to pass a DogId instead of a CatId.
A few years ago, Eishay Smith mentioned during a conversation that he likes to use Java Generics to enforce identifier type safety. He suggested what in Scala would look like:
case class Id[T](id: String)
Using the Id[T] type, I can now use the T type argument to break compilation when passing wrong identifier values:
type CatId = Id[Cat] type DogId = Id[Dog] ... val id: DogId = _ val cat = catRepo.get(id) // this now breaks compilation; expected: CatId, actual: DogId
This is all well and nice, but I still need to be able to create DogId or CatId instances from strings somewhere in the system (for instance, when mapping from a database query). Using the current code, I’ll have to break the abstraction when creating the Id instance, assigning a value of Id[T] to my DogId value:
val id = Id[Dog](resultSet.get("id"))
How can we avoid this? I decided to use a helper trait and create a “companion” object for each type:
case class Id[T](id: String)
trait IdGen[T] {
def apply[T](id: String) = Id[T](id)
}
...
type CatId = Id[Cat]
type DogId = Id[Dog]
object CatId extends IdGen[Cat]
object DogId extends IdGen[Dog]
...
val id = DogId(resultSet.get("id"))
which now allows me to abstract away the creation of Id instances, resulting in a system-wide agnosticism to the fact that my DogId type is actually an Id[Dog].
A final issue that bugs me is that nowhere in the Java ecosystem is to be found a better Guid implementation than java.util.UUID, which contains some terrible code smells hidden inside it. At Wix, we’ve been using a Guid[T] class for some time now, which looks something like this:
case class Guid[T](id: String) extends Id[String] {
override def toString = id.toString
}
object Guid {
def apply[T](id: UUID) = new Guid[T](id.toString)
def random[T] = Guid[T](UUID.randomUUID())
}
trait GuidGen[T] {
def random = Guid.random[T]
def apply(id: String) = Guid[T](id)
}
which now allows me to create Guids without holding reference to instances of Java’s UUID class:
type CatId = Guid[Cat] type DogId = Guid[Dog] object CatId extends GuidGen[Cat] object DogId extends GuidGen[Dog] val dogId = DogId.random val dog = dogRepo.get(dogId)

Recent Comments