If you are using ShopSite Pro, you can enforce this by using the JavaScript CheckIt function located under Commerce Setup -> Order System -> Shopping Cart. Here is some sample code you can use to enforce a maximum quantity of one for a particular item:

ns_count = 0; 
 
 for(ns=0; ns<ss_sku.length; ns++){ 
    if(ss_sku[ns]=='XXX'){ 
         ns_count+= ss_quantity[ns]; 
    } 
 } 
 
 if (button == "8") { 
   if(ns_count > 1){ 
     alert("You may only order one of product YYY, please adjust your quantity accordingly"); 
     
     return false; 
   } 
 }

Replace XXX with the SKU of the product and YYY with the product name. If you want to set a maximum quantity greater than one, simply replace the value of “1″ in the above code with the desired maximum value.