<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	>
<channel>
	<title>Comments on: Nose and Decorators: be careful!</title>
	<atom:link href="http://www.blueskyonmars.com/2008/07/25/nose-and-decorators-be-careful/feed/" rel="self" type="application/rss+xml" />
	<link>http://www.blueskyonmars.com/2008/07/25/nose-and-decorators-be-careful/</link>
	<description>Kevin Dangoor on Creating Software Products</description>
	<pubDate>Thu, 04 Dec 2008 02:50:47 +0000</pubDate>
	<generator>http://wordpress.org/?v=2.6.3</generator>
		<item>
		<title>By: holger krekel</title>
		<link>http://www.blueskyonmars.com/2008/07/25/nose-and-decorators-be-careful/#comment-209816</link>
		<dc:creator>holger krekel</dc:creator>
		<pubDate>Mon, 28 Jul 2008 12:54:55 +0000</pubDate>
		<guid isPermaLink="false">http://www.blueskyonmars.com/2008/07/25/nose-and-decorators-be-careful/#comment-209816</guid>
		<description>FWIW i also think that randomized running of tests is a good idea (in some cases running tests concurrently would also be worthwhile).  With py.test this implicitely already happens when you use the experimental "distribute test to multiple machines or processes" or "loop on failing test set" modes.  Having a more explicit option, though, would be nice.</description>
		<content:encoded><![CDATA[<p>FWIW i also think that randomized running of tests is a good idea (in some cases running tests concurrently would also be worthwhile).  With py.test this implicitely already happens when you use the experimental &#8220;distribute test to multiple machines or processes&#8221; or &#8220;loop on failing test set&#8221; modes.  Having a more explicit option, though, would be nice.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Kevin Dangoor</title>
		<link>http://www.blueskyonmars.com/2008/07/25/nose-and-decorators-be-careful/#comment-209815</link>
		<dc:creator>Kevin Dangoor</dc:creator>
		<pubDate>Mon, 28 Jul 2008 12:08:39 +0000</pubDate>
		<guid isPermaLink="false">http://www.blueskyonmars.com/2008/07/25/nose-and-decorators-be-careful/#comment-209815</guid>
		<description>Well, sure, absence of evidence is not evidence of absence... That's the argument made by functional programming types who want *provably* accurate software (and never ship anything :)

I'll tell you what, though, I will concede that that the cost of not depending on test ordering is just a few lines of code. I'll also agree that that cost is small enough that allowing for random order/individual running is more of a win than the cost.

This does raise an interesting question... has anyone released a nose plugin that does random test ordering?</description>
		<content:encoded><![CDATA[<p>Well, sure, absence of evidence is not evidence of absence&#8230; That&#8217;s the argument made by functional programming types who want *provably* accurate software (and never ship anything <img src='http://www.blueskyonmars.com/wp-includes/images/smilies/icon_smile.gif' alt=':)' class='wp-smiley' /> </p>
<p>I&#8217;ll tell you what, though, I will concede that that the cost of not depending on test ordering is just a few lines of code. I&#8217;ll also agree that that cost is small enough that allowing for random order/individual running is more of a win than the cost.</p>
<p>This does raise an interesting question&#8230; has anyone released a nose plugin that does random test ordering?</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Titus Brown</title>
		<link>http://www.blueskyonmars.com/2008/07/25/nose-and-decorators-be-careful/#comment-209814</link>
		<dc:creator>Titus Brown</dc:creator>
		<pubDate>Mon, 28 Jul 2008 11:59:32 +0000</pubDate>
		<guid isPermaLink="false">http://www.blueskyonmars.com/2008/07/25/nose-and-decorators-be-careful/#comment-209814</guid>
		<description>"I actually don’t tend to run into many bugs that would be caught by reordering my tests…" -- this raises a red flag for me.  Absence of evidence is not evidence of absence!

Anecdote: I once found a bug in my Cartwheel project (probably my biggest public project) that I detected because I could run tests in any order.  The bug was severe but basically invisible; I don't think I would have found it any other way until it had caused data corruption in my database.

More generally, order dependency of tests allows hidden linkages between disparate parts of your program.  These are sources of additional complexity and (therefore) often sources of bugs...

but I'll stop brow-beating you now :)</description>
		<content:encoded><![CDATA[<p>&#8220;I actually don’t tend to run into many bugs that would be caught by reordering my tests…&#8221; &#8212; this raises a red flag for me.  Absence of evidence is not evidence of absence!</p>
<p>Anecdote: I once found a bug in my Cartwheel project (probably my biggest public project) that I detected because I could run tests in any order.  The bug was severe but basically invisible; I don&#8217;t think I would have found it any other way until it had caused data corruption in my database.</p>
<p>More generally, order dependency of tests allows hidden linkages between disparate parts of your program.  These are sources of additional complexity and (therefore) often sources of bugs&#8230;</p>
<p>but I&#8217;ll stop brow-beating you now <img src='http://www.blueskyonmars.com/wp-includes/images/smilies/icon_smile.gif' alt=':)' class='wp-smiley' /></p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Kevin Dangoor</title>
		<link>http://www.blueskyonmars.com/2008/07/25/nose-and-decorators-be-careful/#comment-209813</link>
		<dc:creator>Kevin Dangoor</dc:creator>
		<pubDate>Mon, 28 Jul 2008 11:50:43 +0000</pubDate>
		<guid isPermaLink="false">http://www.blueskyonmars.com/2008/07/25/nose-and-decorators-be-careful/#comment-209813</guid>
		<description>@Holger: that strategy has definitely worked nicely with Paver. I'd go for that rather than true decorators any day.

@Titus: I generally don't care about running individual test cases. Any single test module of mine runs quickly enough that it's not a problem.

And, it only takes a few seconds to communicate to others on my team that the test cases are set up to run in order. (For a public project, there is a difference there...)

I actually don't tend to run into many bugs that would be caught by reordering my tests...


That said, when I switched to py.test/nose/testido (the thing I made before nose came into existence), I stopped using unittest.TestCases because of how annoying they were to work with. The reason that unittest.TestCases are annoying is really only test collection, though. You're right that unittest.TestCases are plenty convenient when working within nose, and I will consider using them the next time I have more fixture I want to build up and tear down.

Just because I don't have to use unittest.TestCase doesn't mean I shouldn't :)</description>
		<content:encoded><![CDATA[<p>@Holger: that strategy has definitely worked nicely with Paver. I&#8217;d go for that rather than true decorators any day.</p>
<p>@Titus: I generally don&#8217;t care about running individual test cases. Any single test module of mine runs quickly enough that it&#8217;s not a problem.</p>
<p>And, it only takes a few seconds to communicate to others on my team that the test cases are set up to run in order. (For a public project, there is a difference there&#8230;)</p>
<p>I actually don&#8217;t tend to run into many bugs that would be caught by reordering my tests&#8230;</p>
<p>That said, when I switched to py.test/nose/testido (the thing I made before nose came into existence), I stopped using unittest.TestCases because of how annoying they were to work with. The reason that unittest.TestCases are annoying is really only test collection, though. You&#8217;re right that unittest.TestCases are plenty convenient when working within nose, and I will consider using them the next time I have more fixture I want to build up and tear down.</p>
<p>Just because I don&#8217;t have to use unittest.TestCase doesn&#8217;t mean I shouldn&#8217;t <img src='http://www.blueskyonmars.com/wp-includes/images/smilies/icon_smile.gif' alt=':)' class='wp-smiley' /></p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Titus Brown</title>
		<link>http://www.blueskyonmars.com/2008/07/25/nose-and-decorators-be-careful/#comment-209805</link>
		<dc:creator>Titus Brown</dc:creator>
		<pubDate>Sat, 26 Jul 2008 14:49:18 +0000</pubDate>
		<guid isPermaLink="false">http://www.blueskyonmars.com/2008/07/25/nose-and-decorators-be-careful/#comment-209805</guid>
		<description>Your post reinforces my belief that relying on the order in which tests are run to make them run correctly is a REALLY BAD IDEA :).

Some reasons why:

 - you can't run individual test cases reliably if they depend on other tests to do the correct thing

 - setup and teardown are what the setup and teardown fixtures are FOR

 - you're violating commonly held expectations of unittest - for better or for worse, people EXPECT test cases to be independent

 - running tests in random order is a great way to find hidden dependencies in the code you are testing.  Such hidden dependencies are a major source of bugs, in my experience.

The real question is this, tho: to re-use test fixtures, why not have the appropriate setup/teardown functions attached to a class, and group the tests by classes?  (There are plenty of other options in both nose and py.test for doing this, but I tend to just use classes.)</description>
		<content:encoded><![CDATA[<p>Your post reinforces my belief that relying on the order in which tests are run to make them run correctly is a REALLY BAD IDEA :).</p>
<p>Some reasons why:</p>
<p> - you can&#8217;t run individual test cases reliably if they depend on other tests to do the correct thing</p>
<p> - setup and teardown are what the setup and teardown fixtures are FOR</p>
<p> - you&#8217;re violating commonly held expectations of unittest - for better or for worse, people EXPECT test cases to be independent</p>
<p> - running tests in random order is a great way to find hidden dependencies in the code you are testing.  Such hidden dependencies are a major source of bugs, in my experience.</p>
<p>The real question is this, tho: to re-use test fixtures, why not have the appropriate setup/teardown functions attached to a class, and group the tests by classes?  (There are plenty of other options in both nose and py.test for doing this, but I tend to just use classes.)</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: holger krekel</title>
		<link>http://www.blueskyonmars.com/2008/07/25/nose-and-decorators-be-careful/#comment-209804</link>
		<dc:creator>holger krekel</dc:creator>
		<pubDate>Sat, 26 Jul 2008 11:22:41 +0000</pubDate>
		<guid isPermaLink="false">http://www.blueskyonmars.com/2008/07/25/nose-and-decorators-be-careful/#comment-209804</guid>
		<description>Hi Kevin.  Interesting note.  I was experimenting the other day about using mocking with py.test.  it reinforces my current idea of rather going for attaching attributes to a test function object and making the test machinery aware of it. cheers, holger</description>
		<content:encoded><![CDATA[<p>Hi Kevin.  Interesting note.  I was experimenting the other day about using mocking with py.test.  it reinforces my current idea of rather going for attaching attributes to a test function object and making the test machinery aware of it. cheers, holger</p>
]]></content:encoded>
	</item>
</channel>
</rss>

<!-- Dynamic Page Served (once) in 0.378 seconds -->
