The Four Horsemen of the Testing Apocalypse

Column info : The Four Horsemen of the Testing Apocalypse by James A. Whittaker.

Food for thought when one is considering a new project or rethinking your test strategy of an old friend.

Warning phases that should trigger reexamination of your test strategy:

• "Go and test it."
• "Go and test it some more."
• "Re-execute the tests."
• "Good news, one million regression tests didn't find any defects."
• "Testers can act like users and find 'customer bugs' before we ship."
• "Testing is boring."

vi: Automatically append closing characters

Stumbled across this blog post that may come in handy some day. Shows some of the advanced capabilities of VIM.
http://vim.wikia.com/wiki/Automatically_append_closing_characters

Don't need this now because I am using the PowerGUI Script Editor to work on my powershell scripts.

Plagiarism, a much larger problem then you though

I bookmarked this email a while ago and just got around to watching it today (1 hour duration).

"I gave a lecture on plagiarism in professional writing and research that some of you might find interesting. The lecture was taped and is available at http://media.fit.edu/load.php?clipid=3102"

The lecture is titled Plagiarism Detection Software by Professor Cem Kaner, 2008. I don't have access to the citations he handed out before the lecture. (Please add a comment to this post if you have a link.)

The following is what I took away and want to be able to refer back to (so I am blogging it).

• Plagiarism = using someone else's words or ideas.
• Copyright = using someone else's words.
• Citation = acknowledgment of someone else's ideas.
• Citation is done to provide the reader with a way to follow up on an idea, dive deeper.
• I find it unbelievable how much plagiarism there it, especially in "professional" venues.
• Plagiarism detection software marketing claims didn't stand up (in 2008), profession/private publications not included.
• Deans tend to think that if the software didn't flag a paper then plagiarism didn't occur.

Plagiarism detection software are tools used by self respecting plagiarist to double check that their cheating will not be routinely caught. The same tool will be used by a university or publisher to check for plagiarism. Sounds like a university tells you which software will be used to do the plagiarism checking so you can fix the paper before submitting. [This was my idea but I give credit to Cem has I think his lecture was designed to lead us to this conclusion.]

I will use citations to show that I know what I am talking about, that I am familiar with current technology/thinking and that I didn't pull these ideas out of my *.

KeyGrinder

So you and all your clan use one password for every stupid site that doesn't matter. I know you use a better/secure/unique p4sSw0rd for your banking sites but how can you possibly remember a different password for every account you have.

KeyGrinder is a program that provides theft-resistant passwords without you having to think much about them. Using the pwdHash algorithm you enter the website URL or account name and your simple easy to remember (and type) key/password. KeyGrinder then produces a reasonably secure password based on the website/account and your simple password. Then take this generated password and use it for the website's (so each of your sites/accounts have a unique password while you remember only one key.)

You end up with each website having a unique secure password while you remain the weak link (because you always use the same simple key to generate these passwords.) In other words, this process isolates risk from website/account security compromisses but will not protect you from angry girlfriends that you once trusted.

Download KeyGrinder from the Apple App Store, for iPhone and iPad. Or use the www.KeyGrinder.com website to generate the same passwords on the fly.

There is little useful documentation on the app but I did fine this blog post on how to use KeyGrinder. If you don't checkout the PwdHash project and think it is cool then most likely you will never seriously use KeyGrinder.

Security is inversely proportional to usability.

MS Windows screen capture

Microsoft's Snipping Tool is part of the more complete Windows 7 packages.

I have been using Lightscreen at work and it meets my needs quite nicely. Just though I'd blog this for the day I am on a machine without my standard testing tools thumb drive.

Variable Naming Conventions

The benefits of naming conventions in software code are generally recognized while specific "conventions" are debatable. Personally I'm looking for the implied intent and meaning of the source code, code that reads as a narrative. Here are the heuristics that I use (when not given a coding standard.)

• Compound identifiers, containing more that one word
• CamelCase the compound identifiers
• Start with a letter; [a-z,A-Z]
• Use only letters and numbers, no other chars like -_+~!. (Under_score is possible exception.)
• Mnemonic names built from a (informal) dictionary of shorthand strings.
• Start function identifiers with a cmdlet verb (Powershell convention).

Since I am currently developing in a .NET world the Microsoft recommendations also apply.

• UpperCamelCase for most identifiers.
• lowerCamelCase for parameters and variables.
• No type prefix hints.
• End the identifier with a base class name (optional).

And then there are the PowershellCommunity.org recommendations.

• ScriptAndGlobalFunctionNamesShouldFollowCmdletNamingStandards
• VerbsShouldBeSelectedFromStandardList
• ScriptAndGlobalFunctionNamesShouldBePascalCased
• CamelCaseVariableNames
• NounNamesShouldBeSpecific
• NounNamesShouldBeSingular
• ScriptsIntendedToBeDotSourcedShouldUseLibraryPrefix
• ProvideDescriptiveVariableNames
• AvoidHungarianNotation

Sail to other shores with Coding Guidelines (PDF), naming conventions of OBO Foundry, Hungarian Notation pros/cons, and Powershell best practices.

An Education in Software Testing

There is no better or insightful instruction on software testing then this black box system testing course. Skip the course introduction and start at 6:30 minutes into the video were the lecture gets going.

The second part of the lecture is a great introduction to software testing. If you are not particularly interested the the course mechanics, skip forward to 2:24 minutes into the video.

Expanding Reliability

My position statement is, Expanding Reliability.

Guy Kawasaki's book Enchantment: The Art of Changing Hearts, Minds, and Actions suggests that your position statement be

• Short
• Clear
• Different
• Humble

Since I work on software that is already "good enough", my position statement communicates that I intend to ever expand the amount of code/functions/stories/edge-cases that work as expected (reliability).

Origins of PowerShell; interview with Bruce Payette

Bruce Payette, one of the original PowerShell team members and author of the Windows PowerShell scripting language here discusses the design goals and origins of the PowerShell language.

His tech history is interesting. Sounds like the kind of guy I use to work with back in the day (before .com world).

Talks about how PowerShell came into existance. Why > wasn't used for greater then, how the language was designed and developed, et cetera.

Bruce's book is perhaps the most respected book on PowerShell for developers, Windows Powershell in Action, Second Edition.

Good Enough, so far

These slogans came to me today while struggling with the transition from 20th century software development methodologies into an agile mindset.

• Never do today what can be put off until tomorrow.
• Good Enough, so far. (User stories will change but remember the above slogan.)

Lightscreen screen capture tool

Stumbed across a nice tool for doing screen captures on a Windows machine, Lightscreen.

You can grab the full screen or a window or a selected rectangle. Then the image is saved in the format you configured in the folder you configured with a filename prefix you configred.

Download it from the Lighscreen site, part of the SourceForge.net project.

Extreme Characters and Soap Opera Testing

I'm reading Mike Cohn's book User Stories Applied: For Agile Software Development in order to switch my mind set into the Agile Development mode for the coming year(s). I recommend this book for people doing agile development.

Extreme Characters is a concept the book introduces. Mike correctly states that it will not be useful for most project but it is worth knowing about. I agree. But it got me thinking. Wouldn't Extreme Characters fit into Soap Opera Testing perfectly.

Cohn sites Djajdiningrat, J.P., W.W. Gaver and J.W. Fres, "Interaction relabelling and Extreme Characters: Methods for exploring aesthetic interactions." Symposium on Designing Interactive Systems 2000, 2000:66-71. See poster for the article.

A 5 minute google didn't turn up much of anything detailed on Extreme Characters other than what Mr. Cohn wrote/teaches. Food for thought and grounds for further research.

(Cool. I bought the Kindle ebook and read in on my laptop at work or on the Kindle at home. It is also sitting on my iPhone should I want to make a quick reference during a stand up.)

Killing iPhone apps

In order to resolve the iPhone Audible app issues one has to completely close the app to be sure it’s still not running on the back end with an error.

1. Double click on the home button, this will bring up the multitask screen, the audible icon should be listed.
2. Press and hold the audible icon. This will give the option to end the task.
3. A minus sign will appear on the screen and you need to tap that.
4. Once this is done, restart your Audible App, and it should now operate properly.

Test Case Design: Absence of evidence vs evidence of absence

Testing for the "absence of something" could be as simple as looking to see if that something is there. However, just because you do not see/find something doesn't mean that it is not there. Furthermore, just because it is not there does not mean that the proceedure to remove something, or prevent something from being there, is working.

Test Case Design

I just got burned by a test case that gave a false positive when it was executed. The verification step was to look for the existance of a secret string in the post-processed log files. I looked in the post-processed logs and and didn't find any evidence of the string thus the test passed.

But, there was a defect and not all log files were instrumented correctly so in a few cases the secret strings were not actually get stripped out of the post-processed logs.

A better test design would be to locate examples of each type of secret string in the raw log files. Then verify that each of these were stripped out by the post-processing. In other words, prove that there is something to be removed before believing that it was removed (and not just missing in the first place.)

User Stories

“As a [type of user], I want [some goal] so that [some reason].”

As a [role], I can [feature] so that [reason].

As a [role], I want [feature] because [reason].

As a [role], I can [feature].

Advantages of this approach. Don't forget to make use of Acceptance Stories as well.

The Testing Geek has a good explanation of User Stories.

Learning PowerShell

My professional goal for 2010 is to start using PowerShell. The developers back in Chicago use it for everything and it seems to be the new scripting language of choice for the Window world. (Pash is a cross platform open source version was started but no news for a couple of years.)

Lots of resources on the net; blogs, script repositories, books, podcasts and such.

I have been reading a tutorial designed for people who don't know object oriented programming. As my scripting skills were developed a century ago. There hasn't been an opportunity to think OOP since the LISP days at IntelliGenetics. This introduction with explanation of OOP terminology/thought is very helpful. I imagine it would be a bit slow for someone versed in .NET or C++. Check it out on the Microsoft site, Sesame Script.

I am using the book Microsoft Windows PowerShell 2.0 Programming for the Absolute Beginner by Jerry Lee Ford to great effect. Reading it cover to cover and doing the example assignments. After completing chapter 3 I felt that I could re-write any script I've created in csh|sh|ksh into PowerShell. So no problem going forward building scripts at work (check off one 2010 job goal;).

Penetration Testing

I am listening to a class from East Tennessee Status University (ETSU) taught by David Frazier, Ethical Hacking CSCI 4957 (iTunes U). This wiki page defines to term Penetration Testing, an active analysis of a system's security primarily by utilizing common ways of compromising the system.

One of these days I would like to go through the class exercises and homework myself. A lot of tools and resources are discussed. I liked the concept of "competative grading", the person who has the best/biggest/most of the assignment gets the top grade and all other get a percentage of that top score.

In the mean time, I vow to improve my personal security by maturing my use of passwords. After learning more I'll investigate the tools. (Can't do it from work because Big Brother doesn't allow access to "potentially malicious" sites, good job IT.)

Read this article, Digital Wallets and Whistle Blowing. What do you think. Good problem for an ethics discussion.

Here is a launching point; Software Engineering Ethics Research Institute.

Good list of penetration test ideas on the SANS site: CWE/SANS Top 25 Most Dangerous Software Errors.

Google Hacking Database

Testing Moving Targets

In some organizations, not anywhere I have ever worked, testers feel they are undervalued and not respected. The vibe is felt when management sends them off to test a moving target knowing that they will be throwing their work out when the new release-candidate version is produced by the hard working software engineers who fixed so many bad bugs to get the product ready. (The programmers didn't do a good job and/or the schedule was bogus so the development schedule had to slip but since the release date can't slip we will have to test harder and faster.)

Sometimes is it tough to dig in deep when you know that the "real product" will be significantly different from what you are testing today.

"If it is going to be changing significantly, then it is not ready for validation testing."—K.

Software Tester Attributes

A watched a Microsoft webinar today about a new test tool integrated into Visual Studio, Test Professional 2010. I like that they are attempting to make a tool for the way most of us work, not asking us to work the way an expensive tool requires. (You had us spend all that money you damn well better use the tool.)

They had a slide in the introduction that listed attributes of a professional software tester that pretty much describes us.

Background & Motivations

• Typically does not have any formal CS training or background [not me]
• Quite often was once a user of the product being tested [not me]
• Sense of achievement comes from making users lives easier
• Feels like the customer advocate
• Likes tracking down hard-to-find bugs
• Competes with developers [don't understand this]
• Has no desire to become a developer [been there, done that, no thanks]

Focus & Approach

• Driven by requirements
• Assessing quality
• Scope is user centered and scenario driven
• Often exploratory or ad-hoc
• Likes to be able to spend time with users
• Finding bugs, not running tests
• Tests defensively [test then code]
• Provides detailed and accurate bug reports
• Dislikes ambiguity

Authoring Tests

• Automation for repetitive tasks
• Elucidate designs
• Hand-on approach to catch unforeseen bugs
• Doesn't believe in 100% automation [$$$ from Nigeria]
• Wants tools that help find bugs
• Creates flexible, reusable tests
• May not always use formal test cases to find bugs
• Always looks for opportunities to find bugs

Password Security

We are always hearing about password security and the need to create passwords for logins all across the internet and at work. But what is a good password? blah blah blah

Password Meter was suggested to me by my insurance company. Read about it or just use it. ;)

Click the image to see the analysis of Bodhr4n +ippEr as a password.

I like it because it lists the requirements that make a strong password and it scores the passwords you enter in real time. I suggest you uncheck "Hide:" so you know that you are typing in is correctly.