visual studio 2012 - vstest.console.exe with /EnableCodeCoverage just "hangs"... how debug, and how to fix? -
i'm using vstest.console.exe (vs2012) run tests /enablecodecoverage, , .runsettings defines "code coverage" datacollector (see codecoverage.runsettings in code block below).
i'm running powershell build script, invokes:
vstest.console.exe /inisolation /logger:trx /enablecodecoverage /settings:codecoverage.runsettings /testcasefilter:"testcategory=customers" bin\release\sdm.test.integtest.dll
previously command working, recent new project integrated old legacy code has brought in lot of new dependencies/dlls.
what see command "hangs", , never seems run of tests. when use sysinternals process explorer see activity in vstest.executionengine.exe ... best guess is attempting instrument whole bunch of dlls .runsettings file should excluded. that's guess.
any in figuring out how diagnose issue appreciated.
codecoverage.runsettings below:
<?xml version="1.0" encoding="utf-8"?> <runsettings> <!-- configurations affect test framework --> <runconfiguration> <!-- path relative solution directory --> <resultsdirectory>.\testresults</resultsdirectory> <!-- [x86] | x64 - can change menu test, test settings, default processor architecture --> <targetplatform>x64</targetplatform> <!-- framework35 | [framework40] | framework45 --> <targetframeworkversion>framework45</targetframeworkversion> </runconfiguration> <!-- configurations data collectors --> <datacollectionrunsettings> <datacollectors> <datacollector friendlyname="code coverage" uri="datacollector://microsoft/codecoverage/2.0" assemblyqualifiedname="microsoft.visualstudio.coverage.dynamiccoveragedatacollector, microsoft.visualstudio.tracecollector, version=11.0.0.0, culture=neutral, publickeytoken=b03f5f7f11d50a3a"> <configuration> <codecoverage> <!-- additional paths search .pdb (symbol) files. symbols must found modules instrumented. if .pdb files in same folder .dll or .exe files, automatically found. otherwise, specify them here. note searching symbols increases code coverage runtime. keep small , local. --> <!-- <symbolsearchpaths> <path>c:\users\user\documents\visual studio 2012\projects\projectx\bin\debug</path> <path>\\mybuildshare\builds\projectx</path> </symbolsearchpaths> --> <!-- include/exclude lists: empty "include" clauses imply all; empty "exclude" clauses imply none. each element in list regular expression (ecmascript syntax). see http://msdn.microsoft.com/library/2k3te2cs.aspx. item must first match @ least 1 entry in include list included. included items must not match entries in exclude list remain included. --> <!-- match assembly file paths: --> <modulepaths> <include> <modulepath>.*\.dll$</modulepath> <modulepath>.*\.exe$</modulepath> </include> <exclude> <modulepath>.*cppunittestframework.*</modulepath> <modulepath>.*[uu]nit[tt]est\.dll</modulepath> <modulepath>.*[ii]nteg[tt]est\.dll</modulepath> <modulepath>.*bomicustomautopoco\.dll</modulepath> <modulepath>.*common\.logging\.dll</modulepath> <modulepath>.*common\.logging\.log4net129.dll</modulepath> <modulepath>.*fluentassertions\.dll</modulepath> <modulepath>.*x509publickeyparser\.dll</modulepath> <modulepath>.*efcachingprovider\.dll</modulepath> <modulepath>.*efproviderwrappertoolkit\.dll</modulepath> <modulepath>.*log4net\.dll</modulepath> <modulepath>.*datahelperdtcbridge\.dll</modulepath> <modulepath>.*\.ni\.dll</modulepath> <modulepath>.*mscorlib\.dll</modulepath> <modulepath>.*vjslib\.dll</modulepath> <modulepath>.*microsoft\..*dll</modulepath> <modulepath>.*system\.enterpriseservices\..*dll</modulepath> <modulepath>.*system\.componentmodel\..*dll</modulepath> <modulepath>.*system\.configuration\..*dll</modulepath> <modulepath>.*system\.core\..*dll</modulepath> <modulepath>.*system\.data\..*dll</modulepath> <modulepath>.*system\.entity\..*dll</modulepath> <modulepath>.*system\.identitymodel\..*dll</modulepath> <modulepath>.*system\.numerics\..*dll</modulepath> <modulepath>.*system\.runtime\..*dll</modulepath> <modulepath>.*system\.servicemodel\..*dll</modulepath> <modulepath>.*system\.transactions\..*dll</modulepath> <modulepath>.*system\.web\..*dll</modulepath> <modulepath>.*system\.xml\..*dll</modulepath> <modulepath>.*msdia110typelib_clr0200\.dll</modulepath> <modulepath>.*vstest.executionengine.exe</modulepath> <modulepath>.*bomi2service\.dll</modulepath> <modulepath>.*nakedobjects\..*dll</modulepath> <modulepath>.*nakedobjects\..*dll</modulepath> <modulepath>.*sdm\.corejava\.dll</modulepath> <modulepath>.*sdm\.datahelper\.dll</modulepath> <modulepath>.*sdm\.events\.dll</modulepath> <modulepath>.*sdm\.infrastructure\.dll</modulepath> <modulepath>.*sdm\.infrastructure\.attributes\.dll</modulepath> <modulepath>.*sdm\.systems\.application\.dll</modulepath> <modulepath>.*sdm\.systems\.distribution\.library\.dll</modulepath> <modulepath>.*sdm\.systems\.distribution\.server\.dll</modulepath> <modulepath>.*sdm\.objectstore\.dll</modulepath> <modulepath>.*sdm\.profiler\.dll</modulepath> <modulepath>.*sdm\.resultsprocessor\.dll</modulepath> <modulepath>.*sdm\.systems\.reflector\.dll</modulepath> <modulepath>.*sdm\.test\.fixtures\.dll</modulepath> <modulepath>.*sdm\.utilities\.dll</modulepath> <modulepath>.*spring\.core\.dll</modulepath> <modulepath>.*techtalk\.specflow\.dll</modulepath> </exclude> </modulepaths> <!-- match qualified names of functions: --> <!-- (use "\." delimit namespaces in c# or visual basic, "::" in c++.) --> <functions> <exclude> <function>^fabrikam\.unittest\..*</function> <function>^std::.*</function> <function>^atl::.*</function> <function>.*::__gettestmethodinfo.*</function> <function>^microsoft::visualstudio::cppcodecoverageframework::.*</function> <function>^microsoft::visualstudio::cppunittestframework::.*</function> </exclude> </functions> <!-- match attributes on code element: --> <attributes> <exclude> <!-- don’t forget "attribute" @ end of name --> <attribute>^system.diagnostics.debuggerhiddenattribute$</attribute> <attribute>^system.diagnostics.debuggernonusercodeattribute$</attribute> <attribute>^system.runtime.compilerservices.compilergeneratedattribute$</attribute> <attribute>^system.codedom.compiler.generatedcodeattribute$</attribute> <attribute>^system.diagnostics.codeanalysis.excludefromcodecoverageattribute$</attribute> </exclude> </attributes> <!-- match path of source files in each method defined: --> <sources> <exclude> <source>.*\\atlmfc\\.*</source> <source>.*\\vctools\\.*</source> <source>.*\\public\\sdk\\.*</source> <source>.*\\microsoft sdks\\.*</source> <source>.*\\vc\\include\\.*</source> </exclude> </sources> <!-- match company name property in assembly: --> <companynames> <exclude> <companyname>.*microsoft.*</companyname> </exclude> </companynames> <!-- match public key token of signed assembly: --> <publickeytokens> <!-- exclude visual studio extensions: --> <exclude> <publickeytoken>^b77a5c561934e089$</publickeytoken> <publickeytoken>^b03f5f7f11d50a3a$</publickeytoken> <publickeytoken>^31bf3856ad364e35$</publickeytoken> <publickeytoken>^89845dcd8080cc91$</publickeytoken> <publickeytoken>^71e9bce111e9429c$</publickeytoken> <publickeytoken>^8f50407c4e9e73b6$</publickeytoken> <publickeytoken>^e361af139669c375$</publickeytoken> </exclude> </publickeytokens> <!-- recommend not change following values: --> <useverifiableinstrumentation>true</useverifiableinstrumentation> <allowlowintegrityprocesses>true</allowlowintegrityprocesses> <collectfromchildprocesses>true</collectfromchildprocesses> <collectaspdotnet>false</collectaspdotnet> </codecoverage> </configuration> </datacollector> </datacollectors> </datacollectionrunsettings> <!-- adapter specific sections --> <!-- mstest adapter --> <mstest> <mapinconclusivetofailed>true</mapinconclusivetofailed> <capturetraceoutput>false</capturetraceoutput> <deletedeploymentdirectoryaftertestruniscomplete>false</deletedeploymentdirectoryaftertestruniscomplete> <deploymentenabled>false</deploymentenabled> </mstest> </runsettings>
the clue find led solution in event viewer, specifically:
.net runtime version 2.0.50727.5477 - failed cocreate profiler. this did led me fix, add:
<startup uselegacyv2runtimeactivationpolicy="true"> <supportedruntime version="v4.0.30319" /> </startup> to vstest.executionengine.exe.config (if running 64bit) or vstest.executionengine.x86.exe.config (if running 32bit).
this works under both vs (when spawned devenv.exe) , command line (when spawned vstest.console.exe)
here's notes got me solution:
the new dlls referenced include pretty old code built against .net 2.0. after lot of searching, i've pieced that:
- vstest.executionengine.exe uses .net 4 profiling stuff dynamic code coverage instrumentation
- the event viewer error message obscure clue vstest.executionengine.exe failing kick off .net 2.0 profiler
- as noted above, fix update .config (that is, vstest.executionengine.exe.config).
it isn't possible add stuff app.config of project containing tests; must go vstest.executionengine's config (because exe running).
other things tried (none of which, ultimately, helped):
diagnostics in registry
computer\hkey_current_user\software\microsoft\visualstudio\11.0\enterprisetools\qualitytools\diagnostics - enabletracing dword = 1
- tracelevel dword = 4
diagnostics in *.config files
for vstest.console.exe, vstest.discoveryengine.*.exe, vstest.executionengine.*.exe
c:\program files (x86)\microsoft visual studio 11.0\common7\ide\commonextensions\microsoft\testwindow added:
<system.diagnostics> <switches> <add name="tptracelevel" value="4" /> </switches> </system.diagnostics> ... causes log files written %temp%
red herring: tried setting environment variables disable
- cor_enable_profiling=0 - complus_profapi_profilercompatibilitysetting=disablev2profiler ... in attempt disable .net 2 profiler, did nothing.
further investigation (using procexp.exe) showed vstest.executionengine.exe sets cor_enable_profiling=1 irrespective of env var.
moreover, cor_profiler's set guid
- for me, cor_profiler={b19f184a-cc62-4137-9a6f-af0f91730165}
- via regedit, hkey_local_machine\software\classes\clsid{b19f184a-cc62-4137-9a6f-af0f91730165}\inprocserver32
- corresponds "c:\program files (x86)\microsoft visual studio 11.0\common7\ide\commonextensions\microsoft\intellitrace\11.0.0\x64\microsoft.intellitrace.profiler.11.0.0.dll"
looking @ event viewer saw info messages .net 4 profiler started:
.net runtime version 4.0.30319.18063 - profiler loaded successfully. profiler clsid: '{b19f184a-cc62-4137-9a6f-af0f91730165}'. process id (decimal): 7700. message id: [0x2507]. the log files in %temp% suggested no issues.
red herring: tried go other way, , enable via env vars:
complus_profapi_profilercompatibilitysetting=enablev2profiler , explicitly set cor_profiler={b19f184a-cc62-4137-9a6f-af0f91730165} made no difference either.
other red herrings
occasional messages in event viewer of form:
engine::notify_process_attach failed exception: session "mtm_7d145e0c-1c26-44b0-89e5-acc448aaae6d" not exist.
vstest.discoveryengine.tptrace.log... error "addprocess : failed addprocess 5" seems irrelevant.
'system.eventhandler`1[microsoft.visualstudio.testtools.execution.sessionstarteventargs]' 'microsoft.visualstudio.coverage.dynamiccoveragedatacollector' i, 2800, 11, 2014/07/01, 10:59:11.875, pckma0419\vstest.discoveryengine.exe, started vangaurd process command line unregister /wildcard /session:mtm_* i, 2800, 11, 2014/07/01, 10:59:11.880, pckma0419\vstest.discoveryengine.exe, add vangaurd process project object w, 2800, 11, 2014/07/01, 10:59:11.882, pckma0419\vstest.discoveryengine.exe, addprocess : failed addprocess 5 i, 2800, 11, 2014/07/01, 10:59:11.882, pckma0419\vstest.discoveryengine.exe, started vangaurd process command line collect /session:mtm_64f33307-c936-469e-b068-482ec0ea45cf /output:"c:\users\danhaywood\appdata\local\temp\mtm_64f33307-c936-469e-b068-482ec0ea45cf\c44e78af-2475-4747-99f3-e0fc3ca41d51\danhaywood_pckma0419 2014-07-01 10_59_11.coverage" /config:
"c:\users\danhaywood\appdata\local\temp\mtm_64f33307-c936-469e-b068-482ec0ea45cf\codecoverage.config" ~~~
blogs consulted along way:
- what 'additional configuration' necessary reference .net 2.0 mixed mode assembly in .net 4.0 project?
- http://blogs.msdn.com/b/dougste/archive/2009/12/30/failed-to-cocreate-profiler.aspx
- http://social.msdn.microsoft.com/forums/en-us/cf079584-54b0-44df-a157-620cc613fca6/failed-to-cocreate-profiler?forum=netfxtoolsdev
- http://blogs.msdn.com/b/davbr/archive/2007/12/11/debugging-your-profiler-i-activation.aspx
- http://bytes.com/topic/c-sharp/answers/813648-net-crashing
- http://social.msdn.microsoft.com/forums/en-us/ab48e868-528d-4d40-b04f-b8a39ba8bf0c/vs2010-on-windows-2008-wrong-setting-of-corprofiler-environment-variable-in-target-process?forum=vstsprofiler
- http://msdn.microsoft.com/en-us/library/dd778910(v=vs.110).aspx
- http://blogs.msdn.com/b/davbr/archive/2009/05/26/run-your-v2-profiler-binary-on-clr-v4.aspx
- http://cvpcs.org/blog/2011-06-23/getting_ncover_and_nunit_to_play_nicely_with_.net_4.0
- force ncover 1.5.8 use v4 framework testdriven.net does?
Comments
Post a Comment