How to convert a PoolByteArray to uint8_t in C++ [NativeScript]

This is a READ-ONLY snapshot of the Godot Community Forums when TwistedTwigleg was the owner, which is hosted for archival reasons only. The current Godot Community Forums can be found here.

Some links may point to external websites, including the current Godot Community Forums. These websites are not associated with TwistedTwigleg.

How to convert a PoolByteArray to uint8_t in C++ [NativeScript]

xero-wolf

Hi guys Is there any way to convert Godot's PoolByteArray to a type that can converted or cast to a uint8_t* pointer?

I'm trying to add some NDI functionality to Godot via gdnative script C++ plugin. NDI is a protocol for sending video and audio over a network. Basically I want to take whatever you see in the viewport and send it via NDI to any NDI device on the network.

Using the NDI SDK it requires you to send it frames as a uint8_t* but Godot outputs image data as a PoolByteArray.

Now I'm able to copy the PoolByteArray by copying each element of the array one by one using a loop. e.g

uint8_t* frame_data =  (uint8_t*)malloc(1920*1080*4);  //used by NDI in another function


//this function is called by on the gdscript end where image_data is the result of vieworpt.get_texture().get_data().get_data()
void NDI::set_video_data(PoolByteArray image_data)
{
	int size = image_data.size();

	for (int i = 0; i < size; i++)
	{
		frame_data[i] = image_data[i]; 
	}
}

This works but naturally it's incredibly slow. Having to copy 8,294,400 elements per frame for a 1920 1080 4 image. Where the 4 is the number of bytes use to represent each pixel in RGBA color format.

My knowledge of C++ is rather limited. Is there any way I could do this conversion better? I'd love to hear and ideas on this thanks.

GlyphTheWolf

Hi,

As I checked from code PoolByteArray should have methods read() and write(). Both return object which seems to have method ptr() which return pointer to data as uint8_t* (const in case of read). Not sure if that will work, but just from short look at the code I would try something like image_data.read().ptr().

xero-wolf

Thank you. Sadly it doesn't seem to work right off bat. I've had to use const_cast<uint8_t*> to convert the pointer image_data.read().ptr() returns and I'm only getting a grey solid as a result. I'm sure it's a step in the right direction tho.

dotted

Using memcpy will speed up transfer considerably, but you need to pass the ptr() value to it.

I would first modify the code you show to fetch the ptr() value and change to using a sampling (such as every 7th or 13th) loop iteration to compare *ptr value vs image_data[i]

The question I would ask is if you are transmitting data as a protocol might expect. Video is rarely sent in a raw/uncompressed format Except for video editing applications, an alpha channel is not used; alpha just isn't a thing in 'broadcast'.

Also note that the image data may not be RGBA format.

PS Write and Read both return a pointer to the same memory. You can use either construct. If Write saves you a cast, use it instead.

Read::FORCE_INLINE const T ptr() const { return this->mem; } Write:: FORCE_INLINE T ptr() const { return this->mem; }

xero-wolf

@dotted said: Using memcpy will speed up transfer considerably, but you need to pass the ptr() value to it.

I would first modify the code you show to fetch the ptr() value and change to using a sampling (such as every 7th or 13th) loop iteration to compare *ptr value vs image_data[i]

The question I would ask is if you are transmitting data as a protocol might expect. Video is rarely sent in a raw/uncompressed format Except for video editing applications, an alpha channel is not used; alpha just isn't a thing in 'broadcast'.

Also note that the image data may not be RGBA format.

PS Write and Read both return a pointer to the same memory. You can use either construct. If Write saves you a cast, use it instead.

Read::FORCE_INLINE const T ptr() const { return this->mem; } Write:: FORCE_INLINE T ptr() const { return this->mem; }

Thank you so much! memcpy(frame_data,image_data.write().ptr(),image_data.size()); Did the trick. Aww man I've been looking for a solution to this far a while now. I did try memcpy before but I was not aware of Read and Write functions at the time. Now I have to see how I can optimize more with your suggestions.

From what I understand NDI does it onw compression of the video data before it is sent over IP. From all the examples shown in the SDK it only receives the frame data as a pointer to the raw image data you provide it. That said there is a 'compress' function I can call on the PoolByteArray but I've have to experiment a bit to see what effect it has if any.

The image color format is indeed RGBA as I've set it as such in another setup function not show in my example and it is the default color format of images in Godot. Generally in broadcast we don't use or need an alpha channel but for graphic overlays I preferred to using a chroma key. Using NDI to animated graphics with alpha channels to virtually anyone on the say network in a dream come true.

Thanks again!

dotted

Ah good. Just be careful about (lossy) compression, it can wreck havoc on multi generational edited video.

I am curious if you considered passing the image data to NDI directly? I would not see a point of rendering faster than compression can go, because at some point it would get back logged.

If you need to have a fresh buffer to render into there may be some way to use double buffered textures (if such a construct exists in godot or if you can render to two different textures) and have rendering in the background while NDI munches on a frame. This way you can save a rather expensive copy. It really comes down to access speeds for textures vs a regular ram copy.

Drewdus42

Have you made any progress in this project? I'm also looking for NDI functionality in Godot.

Pixophir

Guys, be careful. Idk the surrounding code, but it really smells like undefined behaviour. Casting malloc is widely regarded as an error, at least it is superfluous. Why not new [] ? It is C++!. Comparing integers of different signedness and also assigning different types with a potential implicit conversion can bear undefined behaviour.