This one bugged me on one site I was building, but the solution is pretty simple. Just add the following snippet to your functions.php:

// Remove WooCommerce Catalog Ordering Dropdown
remove_action( 'woocommerce_pagination', 'woocommerce_catalog_ordering', 20);