Matlab crashes when the external function in DLL is called (x64 code)

17 views (last 30 days)
I have a code in assembly language that includes few functions. The code was translated into .DLL (x64). Now I want to call the function in x64 Matlab, but when I do so, Matlab crashes.
I use fastcall (as it's the only calling convention for x64), header file is adjusted to fastcall also.
No call function or load library errors, it just crashes.
Any help appreciated. Thanks!

Accepted Answer

Samuel SLenker
Samuel SLenker on 24 Mar 2017
So I found a mistake in my assembly code. Now when I use the new version, Matlab crashes again but with crash dialog where it says on which address of the DLL is the problem. It always crashes at the first instruction that uses two 128 bit registers (xmm). I tried to remove the line, and it again crashed at the next instruction. And then again.
Have no idea where's the problem because in .exe file + dbg the code works fine.
Samuel SLenker
Samuel SLenker on 13 Apr 2017
Yeah, I knew about that need to save registers, but somehow I forgot about it. Well, after saving all the callee-save registers, and of course popping them at the end, Matlab works. Thank you Philip!

Sign in to comment.

More Answers (6)

Jan on 22 Mar 2017
Well, it sounds like the code of the DLL is buggy. Do you have any evidence that this assembler code works?

Samuel SLenker
Samuel SLenker on 22 Mar 2017
Edited: Samuel SLenker on 22 Mar 2017
Hello Jan,
the code should be OK. Before I even moved to Matlab, I went through several tests and optimizations.
I created .exe that calls the function from the .dll and it works without any problems. Tested through debugger. No errors, program runs as expected, like I said, I was optimizing and fixing errors (it took days due to many differences between x86 and x64) in DBG before I moved to Matlab.
I can attach files here once I reach my PC.

Philip Borghesani
Philip Borghesani on 22 Mar 2017
Sounds like a job for the debugger. Without specifics we can't be of much help. The 64bit abi is sometimes called fastcall and sometimes not. LOADLIBRARY does not need or desire the function to be decorated with the microsoft extension "fastcall" decoration. I assume you are doing this on Windows?
Matlab makes some assumptions about processor state that can be violated by assembly code, my first thought is why are you using assembly? If you change any processor settings make sure you put them back before returning to MATLAB...
The more of the flowing information you supply the more likely we are to be helpful:
  • Matlab version and OS.
  • Function source or shortened example source if possible.
  • Header file contents.
  • Matlab example code used to call function.
  • Information on how library was built and example c test code you used to test the library function.
  • If you cant supply enough code to allow reproduction at least give us the stack trace so we can reason about why things are crashing.

Samuel SLenker
Samuel SLenker on 22 Mar 2017
Edited: Samuel SLenker on 22 Mar 2017
Hi Philip,
find the details below:
  • Matlab R2015b x64, Win7 x64
  • SOURCE CODE OF THE DLL IS IN ATTACHMENT (notes are non-english, just ignore them)
  • Script in Matlab (notes are non-english, just ignore them)
clear all; clc
disp('Computation time in Matlab:')
%number of rounds the function is called
plot(y * sqrt(2/N),'r')
hold on
% load lib
hfile1 = 'kody64.h';
[notfound,warnings]=loadlibrary('kody.dll', hfile1,'mfilename','kody_mx');
pM = libpointer('singlePtr', M);
px = libpointer('singlePtr', x);
% call vxm_sse
disp('SSE computation:')
for i=1:Np
y0=zeros(1,N); py0 = libpointer('singlePtr', y0);
calllib('kody', 'vxm_sse', pM, px, py0, length(x)/4);
y_sse1=get(py0, 'Value');
plot(y_sse1 * sqrt(2/N),'g')
hold off
unloadlibrary kody
legend('DCT IV jako maticové násobení v Matlabu','DCT IV pomocí funkce SSE1','DCT IV pomocí funkce SSE2','DCT IV pomocí funkce SSE3','DCT IV pomocí funkce AVX1','DCT IV pomocí funkce AVX2','DCT IV pomocí funkce AVX3',1)
  • Header file is "kody64.h" and looks following:
void vxm_sse(float *a,float *b,float *c,int d);

Jan on 22 Mar 2017
Edited: Jan on 22 Mar 2017
void vxm_sse(float *a,float *b,float *c,int d);
calllib('kody', 'vxm_sse', pM, px, py0, length(x)/4);
While pM, px and py0 are float *, length(x)/4 replies a double, not an int. Let's assume that the int has 64 bits, then please try:
calllib('kody', 'vxm_sse', pM, px, py0, int64(length(x)/4));
calllib('kody', 'vxm_sse', pM, px, py0, int32(length(x)/4));
I assume that this should be changed also:
% y0=zeros(1,N); py0 = libpointer('singlePtr', y0);
y0 = zeros(1, N, 'single'); py0 = libpointer('singlePtr', y0);
Otherwise you provide a single pointer to double data. This should not crash, but reply messed up data.

Samuel SLenker
Samuel SLenker on 22 Mar 2017
The same script was used for x32 DLL in x32 Matlab and it worked without any problems. Of course, the .h file was defined with _stdcall.
None of the changes you mention above was an error. It worked fine. However, I tried to implement your suggestments and Matlab does the same as before - just crashes, no errors, no warnings.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!