GPU CUDA error "An unexpected error occurred trying to launch a kernel. The CUDA error was: invalid configuration argument" resolves automatically on second try
Show older comments
Hi,
I'm trying to run code on GPU and I get the following error:
An unexpected error occurred trying to launch a kernel. The CUDA error was:
invalid configuration argument
When I insert a breakpoint at the line that gives this error and run the code manually I get the same error but if I immediately try running it like again the code runs without the error. This proves that my code is not buggy and it is something else that's causing it to crash.
I have seen the same CUDA error using the NaN and subref functions. If it helps at all, here are the exact lines both of which give the same error on first try, but not on second try:
% On first try:
K>> W0(:,ones(1,size(dWU0,3)),:)
Error using gpuArray/subsref
An unexpected error occurred trying to launch a kernel. The CUDA error was:
invalid configuration argument
% Here it is on second try, immediately following the last one (no actions in between):
K>> W0(:,ones(1,size(dWU0,3)),:)
ans(:,:,1) =
0.0087 0.0087
0.0135 0.0135
0.0202 0.0202
0.0303 0.0303
0.0436 0.0436
0.0667 0.0667
0.0973 0.0973
0.1273 0.1273
0.1582 0.1582
0.1935 0.1935
0.2348 0.2348
0.2767 0.2767
0.3009 0.3009
0.2821 0.2821
0.2036 0.2036
0.0705 0.0705
-0.0867 -0.0867
-0.2266 -0.2266
-0.3169 -0.3169
-0.3459 -0.3459
-0.3220 -0.3220
-0.2701 -0.2701
-0.2121 -0.2121
-0.1611 -0.1611
-0.1206 -0.1206
-0.0881 -0.0881
-0.0612 -0.0612
-0.0389 -0.0389
-0.0218 -0.0218
-0.0103 -0.0103
-0.0032 -0.0032
0.0013 0.0013
0.0054 0.0054
0.0086 0.0086
0.0112 0.0112
0.0124 0.0124
0.0128 0.0128
0.0093 0.0093
0.0054 0.0054
0.0059 0.0059
0.0048 0.0048
0.0045 0.0045
0.0040 0.0040
0.0039 0.0039
0.0039 0.0039
0.0039 0.0039
0.0039 0.0039
0.0038 0.0038
0.0037 0.0037
0.0037 0.0037
0.0037 0.0037
0.0036 0.0036
0.0037 0.0037
0.0037 0.0037
0.0038 0.0038
0.0039 0.0039
0.0038 0.0038
0.0037 0.0037
0.0034 0.0034
0.0033 0.0033
0.0030 0.0030
ans(:,:,2) =
0.0620 0.0620
0.0931 0.0931
0.1302 0.1302
0.1729 0.1729
0.2109 0.2109
0.2376 0.2376
0.2496 0.2496
0.2512 0.2512
0.2390 0.2390
0.1930 0.1930
0.0968 0.0968
-0.0537 -0.0537
-0.2302 -0.2302
-0.3746 -0.3746
-0.4279 -0.4279
-0.3749 -0.3749
-0.2513 -0.2513
-0.1257 -0.1257
-0.0487 -0.0487
-0.0268 -0.0268
-0.0308 -0.0308
-0.0354 -0.0354
-0.0321 -0.0321
-0.0256 -0.0256
-0.0231 -0.0231
-0.0242 -0.0242
-0.0249 -0.0249
-0.0207 -0.0207
-0.0136 -0.0136
-0.0073 -0.0073
-0.0027 -0.0027
0.0023 0.0023
0.0103 0.0103
0.0158 0.0158
0.0197 0.0197
0.0195 0.0195
0.0170 0.0170
0.0145 0.0145
0.0147 0.0147
0.0094 0.0094
0.0094 0.0094
0.0077 0.0077
0.0048 0.0048
0.0052 0.0052
0.0047 0.0047
0.0049 0.0049
0.0048 0.0048
0.0043 0.0043
0.0033 0.0033
0.0027 0.0027
0.0026 0.0026
0.0023 0.0023
0.0025 0.0025
0.0032 0.0032
0.0044 0.0044
0.0053 0.0053
0.0060 0.0060
0.0057 0.0057
0.0045 0.0045
0.0032 0.0032
0.0016 0.0016
ans(:,:,3) =
-0.0826 -0.0826
-0.0976 -0.0976
-0.1043 -0.1043
-0.0936 -0.0936
-0.0674 -0.0674
-0.0228 -0.0228
0.0267 0.0267
0.1056 0.1056
0.1959 0.1959
0.2812 0.2812
0.3282 0.3282
0.3057 0.3057
0.2109 0.2109
0.0744 0.0744
-0.0401 -0.0401
-0.0647 -0.0647
0.0084 0.0084
0.1415 0.1415
0.2627 0.2627
0.3130 0.3130
0.2663 0.2663
0.1646 0.1646
0.0571 0.0571
-0.0321 -0.0321
-0.0928 -0.0928
-0.1287 -0.1287
-0.1545 -0.1545
-0.1726 -0.1726
-0.1818 -0.1818
-0.1817 -0.1817
-0.1754 -0.1754
-0.1644 -0.1644
-0.1537 -0.1537
-0.1374 -0.1374
-0.1179 -0.1179
-0.1006 -0.1006
-0.0818 -0.0818
-0.0607 -0.0607
-0.0417 -0.0417
-0.0325 -0.0325
-0.0220 -0.0220
-0.0142 -0.0142
-0.0041 -0.0041
-0.0028 -0.0028
-0.0012 -0.0012
-0.0020 -0.0020
-0.0022 -0.0022
-0.0012 -0.0012
0.0008 0.0008
0.0026 0.0026
0.0037 0.0037
0.0041 0.0041
0.0036 0.0036
0.0031 0.0031
0.0029 0.0029
0.0032 0.0032
0.0031 0.0031
0.0034 0.0034
0.0032 0.0032
0.0024 0.0024
0.0019 0.0019
% Here's another line of my code showing the same strange behavior:
% First try:
K>> y = nan(s,'like',x)
Error using gpuArray/nan
An unexpected error occurred trying to launch a kernel. The CUDA error was:
invalid configuration argument
% Second try:
K>> y = nan(s,'like',x)
y =
NaN
This is very perplexing to me. Any thoughts?
5 Comments
Ganesh Regoti
on 22 Oct 2019
Hi Sia,
Can I get the following information which would be helpful in finding the issue.
- Which Driver version you are using?
- Which toolkit and it version if you are using one.
- The sizes of the data structures you are using.
- Whether you are using GPUarray or GPUCoder
- If you could share the script /function that would give more insight.
Jai Bhagat
on 26 Feb 2020
Hi Sia, I had the same issue running kilosort2. Did you figure out a fix in the end? I saw something similar was posted as an issue to kilosort2's github, but Marius closed the issue due to inactivity.
Y.Yang
on 23 Jun 2020
I have the same problem when using permute with gpuArray matrix.
Clay Smyth
on 10 Nov 2020
Have there been any solutions? Running into the issue with kilosort 2.
Udaysankar Chockanathan
on 23 Nov 2020
I'm also getting this issue, but only for certain recordings.
Answers (1)
Ben
on 5 Aug 2021
0 votes
Maybe you can consider trying to reduce the size of W0. I encountered this problem before, and it won't happen again until I reduce the size.
Categories
Find more on GPU Computing in Help Center and File Exchange
Products
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
