User-agent: * Allow: / Allow: /wp-admin/admin-ajax.php Allow: /media/ # Disallowed Sub-Directories Disallow: /readme.html Disallow: /lisence.txt Disallow: /content/ Disallow: /client/ Disallow: /wp-admin/ Disallow: /wp-includes/