Unit testing Angular directives is not a simple thing to do. You have to create an HTML string which invokes your directive, run it through angular.element to convert that string into an element, then manually call the $compile service to Angular-ify the whole thing. After that, you have to do some DOM walking wizardry to test that the things that should/shouldn't change have/haven't changed. Then, if your widget contains user interaction behavior, you need to simulate that interaction and check the DOM again. Rinse, repeat, reflux. All because you want to test your directive, which usually boils down to testing its controller.
To be fair, when you compile an element, you pass in a scope, and thus you can inspect the scope for changes that would be reflected in the DOM later. But that still means that you have to fiddle with HTML a little bit. At best it's more complicated to reason about than, say, testing a regular old controller.
I've personally struggled with this particular issue for a while now. I know Angular developers that just don't bother testing directives at all because it seems to them to be much more trouble than the value they get from doing it, even a friend of mine who is absolutely militant about test-driven development. He won't write a single line of code that won't make a failing test pass...unless testing that code involves compiling a directive.
There are two strategies I've used to reduce the friction between me and my directive tests.