c# - Best way to work on 15000 work items that need 1-2 I/O calls each -


i have c#/.net 4.5 application work on around 15,000 items independent of each other. each item has relatively small cpu work (no more few milliseconds) , 1-2 i/o calls wcf services implemented in .net 4.5 sql server 2008 backend. assume queue concurrent requests can't process quick enough? these i/o operations can take anywhere few milliseconds full second. work item has little more cpu work(less 100 milliseconds) , done.

i running on quad-core machine hyper-threading. using task parallel library, trying best performance machine can little waiting on i/o possible running operations asynchronously , cpu work done in parallel.

synchronously, no parallel processes , no async operations, application takes around 9 hours run. believe can speed under hour or less not sure if going right way.

what best way work per item in .net? should make 15000 threads , have them doing work context switching? or should make 8 threads (how many logical cores have) , go way? on appreciated.

my usuall suggestion tpl dataflow.

you can use actionblock async operation , set parallelism high need be:

var block = new actionblock<workitem>(wi => {     dowork(wi);     await task.whenall(dosomeworkasync(wi), dootherworkasync(wi)); },  new executiondataflowblockoptions{ maxdegreeofparallelism = 1000 });  foreach (var workitem in workitems) {     block.post(workitem); }  block.complete(); await block.completion;

that way can test , tweak maxdegreeofparallelism until find number fits specific situation most.

for cpu intensive work having higher parallelism cores doesn't help, i/o (and other async operations) if cpu intensive work short go @ least 1000.


Comments

Post a Comment

Popular posts from this blog

Magento/PHP - Get phones on all members in a customer group -

i working on custom magento extension. working around adminhtml part , i've created custom form there. here form code: <?php class vivasindustries_smsnotification_block_adminhtml_sms_sendmass_edit_form extends mage_adminhtml_block_widget_form { public function _preparelayout() { $extensionpath = mage::getmoduledir('js', 'vivasindustries_smsnotification'); $head = $this->getlayout()->getblock('head'); $head->addjs('jquery.js'); $head->addjs('vivas.js'); return parent::_preparelayout(); } protected function _prepareform() { $form = new varien_data_form(array( 'id' => 'edit_form', 'action' => $this->geturl('*/*/save', array('id' => $this->getrequest()->get...
Read more

php - Bypass Geo Redirect for specific directories -

i'm trying bypass redirect specific url within site. wondering if might able help. the code below code use, original urls replaces example.com i'd able access http://example.com/admin without redirect occuring. is there way this, , there way using own ip? function country_geo_redirect() { $country = getenv('http_geoip_country_code'); if ($country == "us" ) { wp_redirect("http://usa.example.com/"); exit; } else if ($country == "gb" ) { wp_redirect("http://uk.example.com/"); exit; } } add_action('init', 'country_geo_redirect'); you can bypass redirection code if server found ip address. if ip address == <my ip address> ' nothing here else country_georedirect() end if
Read more

php - .htaccess mod_rewrite for dynamic url which has domain names -

i want convert url ex:( example.com/link.php?url=facebook.com ) ( example.com/facebook.com.php ) how can using .htaccess. or there other process make job easier ? i tried adding script .htaccess file options +followsymlinks rewriteengine on rewriterule url-(.*)\.htm$ link.php?url=$1 but converting url adding "url-" ( example.com/url-facebook.com.php ) i want remove "url-" , want make ( example.com/facebook.com.php ) rewriteengine on rewritecond %{the_request} ^get.*link\.php\?url\= [nc] rewritecond %{request_uri} !/system/.* [nc] rewriterule (.*?)link\.php\?url\=/*(.*) /$1$2 [r=301,ne,l] not sure how above work edit of used use remove index.php url
Read more