what is meant by third party platform evaluation

Today I got into an interesting conversation with one of my colleagues(Bala Dutt) about what exactly are we supposed to do when evaluating a third party(open source or commercial) platform. So here it is...

1. Write down all the requirements(as precisely as possible) for which you're evaluating the solution.


2. If possible, classify the above requirements into moscow*


3. Test the solution against above requirements on some dummy data (here we actually use,install and run the software or use the code or whatever, the platform to solve some problems which correspond to our requirements)


4. Now we should be able to give some subjective/objective comments against each requirement. Basically its the gap analysis i.e. does this solution satisfy our requirements out of the box or is there a gap?, if there is a gap then what is the effort needed to fill** that gap?


5. One more thing we can look at how flexible, extensible is it to satisfy any future changes.


6. Above information should be enough to make the call :) .

* moscow = must have, should have, could have and wishes
** filling the gap can sometimes also mean negotiation with the project management :)