Using TCPIP, snapshots of webcam are not being sent from server APP to client.

1 view (last 30 days)
Garabani
Garabani on 27 Nov 2021
Commented: Stephen on 1 Dec 2021
I want to send my webcam live streaming to client using tcpip, so far I have been able to send the webcam from the servers side using this code.
function VIDSwitchValueChanged(app, event) %toggle switch
value = app.VIDSwitch.Value;
tcpipServer = tcpip('0.0.0.0', 30000,'NetworkRole','Server'); %tcpserver doesnt work for me
if strcmp(value, 'On')
app.cam = webcam;
frame = snapshot(app.cam);
data = im2double(frame);
s = whos('data');
s.size;
s.bytes;
set(tcpipServer,'OutputBufferSize',s.bytes);
fopen(tcpipServer);
theSpeakerImage = image(app.UIAxes, zeros(size(frame), 'uint8'));
axis(app.UIAxes, 'image');
preview(app.cam, theSpeakerImage);
fwrite(tcpipServer,data(:),'double');
but I have not been able to read it from my client side while trying to unpack it. So far, this is the best I have reached without producing any error. However, no image or video is seen on client side and when I turn off the app, my server app gets stuck with ctrl+c not working
function Switch_2ValueChanged(app, event)
value = app.Switch_2.Value;
tcpipClient = tcpip('0.0.0.0', 30000,'NetworkRole','Client');
set(tcpipClient,'InputBufferSize',300000);
%set(tcpipClient,'Timeout',30);
if strcmp(value, 'On')
fopen(tcpipClient);
get(tcpipClient, 'BytesAvailable');
tcpipClient.BytesAvailable;
DataReceived =[];
while (get(tcpipClient, 'BytesAvailable') > 0)
tcpipClient.BytesAvailable
rawData = fread(tcpipClient,30000/8,'double');
DataReceived = [DataReceived; rawData];
reshapedData = reshape(DataReceived,650,600,3);
imshow(reshapedData)
pause(0.1)
I have tried imshow since I tried to send the frame through tcpip and treated it like an image, still no luck. How do I go about sending the webcam feed from server side to client side? preferably using plot(app.UIAxes) to preview that image from the clients axis.
sorry for posting only a snippet of the code, trying to attach the mfiles fails every time for some reason.
I have also tried the follwoing for clients side and still no luck
fprintf('0.0.0.0', '%d ', ndims(theSpeakerImage)); %no luck here
DataReceived = [DataReceived; rawData];
DataReceived = reshape(DataReceived,[1,1]); %throws error that the size of the data can not be changed by adding new elements
preview(rawData, theSpeakerImage); %also nothing
plot(app.UIAxes,rawData);% no results

Accepted Answer

yanqi liu
yanqi liu on 29 Nov 2021
yes,may be use http to build server and client process
  3 Comments

Sign in to comment.

More Answers (1)

Walter Roberson
Walter Roberson on 29 Nov 2021
Each time through, you should construct a buffer that contains the image size (list of 3 doubles) followed by the image data. fwrite that to the socket.
on the receiving side, fread 3 double. That will be the image size. Now fread with that size and '*double' precision.
You might find that in practice you need to break the sending data into smaller chunks. If so then still have that buffer I mentioned, but fwrite smaller portions of it.
If you do that then on the receiving side have a small state machine. You start waiting for an image. loop fetching bytes available into a variable and fread that many bytes as *uint8, and put onto the end of your current input buffer.
If you are in the first state and there are less than 24 bytes in the buffer, loop back to waiting for bytes available.
If you are in the first state and your buffer has more than 24 bytes then typecast the first 24 as double to find the current image size, and remove those 24 from the buffer and calculate the product times 8... that's the number of bytes you need for the image. Enter the second state, where you are waiting for data to finish.
If you are in the second state and your buffer is less than the number of bytes you need then loop back to waiting for bytes available.
If you are in the second state and you have enough bytes, then pull the appropriate number of bytes from the front of the buffer and typecast to double and reshape to the size read. Then remove those bytes from the buffer. Display the image and change to the first state and loop back.
This needs a minor flow change such that if you are state 1 and there are at least 24 bytes in the buffer, you can skip waiting for bytes to be available and more on to extracting the size information and switching to state 2. Likewise at state 2 if you already have enough bytes you do not need to listen for more right now. This minor revision to what I described first makes it easier to deal with streams of bytes in the buffer, where you might already have received some of what you need for the next image.
You need to decide on a protocol to signal end of stream. That might be, for example, sending 0 as the image size.
Is there any particular reason why you want to send double precision images? Most webcam images are 8 bits per pixel, 16 at most, so double precision is usually a waste.
  4 Comments
Garabani
Garabani on 30 Nov 2021
Thank you for taking your time time, here is my code, Im running it in App Designer.
for server
classdef WEBCAM_FOR_DOCTOR < matlab.apps.AppBase
% Properties that correspond to app components
properties (Access = public)
UIFigure matlab.ui.Figure
UIAxes matlab.ui.control.UIAxes
VIDSwitch matlab.ui.control.Switch
end
im using a toggle switch to plot the webcam on UIAxes.
% Value changed function: VIDSwitch
function VIDSwitchValueChanged(app, event)
value = app.VIDSwitch.Value;
tcpipServer = tcpip('127.0.0.1', 20000,'NetworkRole','Server');
if strcmp(value, 'On')
app.cam = webcam;
frame = snapshot(app.cam);
data=im2double(frame);
s = whos('data');
s.size;
s.bytes;
pic = buffer(5,5,2);
set(tcpipServer,'OutputBufferSize',s.bytes);
fopen(tcpipServer);
theSpeakerImage = image(app.UIAxes, zeros(size(frame), 'uint16'));
axis(app.UIAxes, 'image');
preview(app.cam, theSpeakerImage);
% assignin('base','var',frame)
fwrite(tcpipServer,pic,'double');
% fwrite(tcpipServer,data(:),'double');
else
fclose(tcpipServer);
closePreview(app.cam);
delete(app.cam);
cla(app.UIAxes);
end
For client side.
classdef WEBCAM_FOR_Patient < matlab.apps.AppBase
% Properties that correspond to app components
properties (Access = public)
UIFigure matlab.ui.Figure
Switch_2 matlab.ui.control.Switch
UIAxes matlab.ui.control.UIAxes
end
Using another swithc (Switch_2) to open tcpip for client side. trying to read the webcam from server to this sides UIAxes to act like a live stream
% Value changed function: Switch_2
function Switch_2ValueChanged(app, event)
value = app.Switch_2.Value;
tcpipClient = tcpip('127.0.0.1', 20000,'NetworkRole','Client');
set(tcpipClient,'InputBufferSize',300000);
%set(tcpipClient,'Timeout',30);
if strcmp(value, 'On')
fopen(tcpipClient);
get(tcpipClient, 'BytesAvailable');
tcpipClient.BytesAvailable;
%trying this code currently
DataReceived =[];
while (get(tcpipClient, 'BytesAvailable') > 0)
tcpipClient.BytesAvailable
rawData = fread(tcpipClient,30000/8,'double');
DataReceived = [DataReceived; rawData];
reshapedData = reshape(DataReceived,650,600,3);
imshow(reshapedData)
pause(0.1)
if strcmp(value, 'Off')
break;
end
end
end
if strcmp(value, 'Off')
fclose(tcpipClient);
end
%was trying these ones before
% theSpeakerImage = image(app.UIAxes, zeros(size(rawData), 'uint8'));
% fprintf('127.0.0.1', '%d ', ndims(theSpeakerImage));
%DataReceived = [DataReceived; rawData];
%DataReceived = reshape(DataReceived,[1,1]);
%imshow(rawData, 'parent', app.UIAxes);
% preview(rawData, theSpeakerImage);
%plot(app.UIAxes,rawData);
% imshow(reshapedData);
% pause(0.1)
% raw = fread(tcpipClient);
%
% rawData = uint8(raw);
% %rawData = reshape(raw,1);
% % theSpeakerImage = image(app.UIAxes, zeros(size(rawData), "int8"));
% % axis(app.UIAxes, 'image');
% % preview(rawData, theSpeakerImage);
% % obj=VideoReader(rawData);
% % vid=read(obj);
% % firstframe=read(obj,1);
% % imshow(firstframe,'video',app.UIAxes);
% v = VideoWriter(rawData);
% video = read(v);
% imshow(video,'video',app.UIAxes);

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!