I can't change the size of the reserved CMA memory in the Kernel

joserp93 · December 14, 2022, 12:49pm

Hi Everyone, I have my image for Nerves based on this 32bit rerelease, because I have to use Gstreamer with Omx.

The detail I need to increase the size of the CMA memory, but I have not been able to.

I have tried this:
Boot the kernel with dtoverlay=vc4-fkms-v3d,cma-256 and append in the cmdline.txt file ‘cma=256M’

But both cases have failed.

cmd("dmesg | grep -i cma")
[    0.000000] cma: Failed to reserve 256 MiB
[    0.000000] Kernel command line: coherent_pool=1M 8250.nr_uarts=1 cma=64M cma=256M  smsc95xx.macaddr=E4:5F:01:63:91:2A vc_mem.mem_base=0xec00000 vc_mem.mem_size=0x10000000  dwc_otg.lpm_enable=0 console=tty1 console=ttyS0,115200 fbcon=scrollback:1024k root=/dev/mmcblk0p2 rootfstype=squashfs rootwait consoleblank=0
[    0.000000] Memory: 3171444K/3211264K available (6144K kernel code, 384K rwdata, 1248K rodata, 2048K init, 411K bss, 39820K reserved, 0K cma-reserved, 3080192K highmem)
[    5.441143] vc_sm_cma: module is from the staging directory, the quality is unknown, you have been warned.
[    5.452259] bcm2835_vc_sm_cma_probe: Videocore shared memory driver

cmd("dmesg | grep -i cma")
[ 0.000000] cma: Failed to reserve 256 MiB
[ 0.000000] Kernel command line: coherent_pool=1M [8250.nr](http://8250.nr/)_uarts=1 cma=64M bcm2708_fb.fbwidth=0 bcm2708_fb.fbheight=0 bcm2708_fb.fbswap=1 smsc95xx.macaddr=E4:5F:01:63:91:2A vc_mem.mem_base=0xec00000 vc_mem.mem_size=0x10000000 dwc_otg.lpm_enable=0 console=tty1 console=ttyS0,115200 fbcon=scrollback:1024k root=/dev/mmcblk0p2 rootfstype=squashfs rootwait consoleblank=0 cma=256M
[ 0.000000] Memory: 3171444K/3211264K available (6144K kernel code, 384K rwdata, 1248K rodata, 2048K init, 411K bss, 39820K reserved, 0K cma-reserved, 3080192K highmem)
[ 5.320059] vc_sm_cma: module is from the staging directory, the quality is unknown, you have been warned.
[ 5.330652] bcm2835_vc_sm_cma_probe: Videocore shared memory driver

Investigating a little more the problem seems to be in the already compiled file: bcm2711-rpi-4-b.dts

That contains some lines of preference where the memory is reserved in 64M.

I found this repository, it seems that is how this compiled file is formed. MZDPI/bcm2711-rpi-4-b.sh at master · tianyoujian/MZDPI · GitHub

If you know about this and can help me it would be very useful.

fhunleth · December 15, 2022, 2:33pm

There was some work on this issue on Slack, so I wanted to summarize it here:

You received some great help on the RaspberryPi Linux issue that you posted.
Upgrading to Linux 5.4 seems like the best way forward.
On Nerves for the Raspberry Pi, upgrading the Linux kernel means modifying the nerves_defconfig to update BR2_LINUX_KERNEL_CUSTOM_TARBALL_LOCATION and BR2_PACKAGE_RPI_FIRMWARE_CUSTOM_VERSION. After figuring out the tags or git hashes to use, you may need to update the Linux configuration since sometimes options change.
To figure out the tag or hash, go to GitHub - raspberrypi/linux: Kernel source tree for Raspberry Pi-provided kernel builds. Issues unrelated to the linux kernel should be posted on the community forum at https://forums.raspberrypi.com/. The Linux 5.4 branch is rpi-5.4y. I prefer to use tags, but they’re a little hard to map to Linux versions since they use dates. I just looked, though, and raspberrypi-kernel-1.20200717-1 is a 5.4 kernel tag!
Updating Linux kernels can be tricky with the Raspberry Pi due to Linux configuration changes. For a 32-bit RPi4, I might take hints from the nerves_system_rpi3 updates. On Slack, I recommended updating Linux version by version if nothing works, so upgrade to Linux 5.0, then 5.1, etc. This is tedious, but I’ve done it when frustrated.
To make it easier to iterate when building and reconfiguring Linux kernels, I’d recommend looking at nerves_systems. Modify the config.exs to only refer to your nerves_system_rpi4 fork.

I think this summaries it. I wish you good luck on this!

joserp93 · December 15, 2022, 10:46pm

Hi Frank.

I chose to go for option 2, go to kernel 5.4 but in 32bits because I need the Omx to record videos. Take as a base the last stable version of Nerves for rpi4 with Kernel 5.4 → v1.21.1 and try to pass it to 32bits.

But I got this error:

I share the repository with the changes:

I am using the indicated dependencies:

nerves_system_br v1.17.1
Buildroot 2021.08.1
Erlang/OTP 24.1.2

If you can take a look at it, it would be a great help.

fhunleth · December 16, 2022, 3:54am

It looks like you haven’t installed build-essential since g++ wasn’t found. There are quite a few other packages that probably need to be installed as well. Since you’re probably going to need to make a few more iterations on the Nerves system to get OMX to work, I’d recommend working through the nerves_systems install instructions and going from there. At the very least, you’ll have pretty much every package installed for any kind of Nerves/Buildroot work.

I took a look at the GitHub repository that you posted. The Linux configuration was really close. I sent a PR and a copy of the Circuits Quickstart firmware that I tried out to test it.

Frank

joserp93 · December 19, 2022, 9:07pm

Hi Frank, I hope you are well this time.

We are super grateful for your help. We have been testing and here are the details.

The version with kernel 5.4 that you sent us compiled without problem, we added the remaining packages to get what we wanted, but we still have the same problem.

We cannot increase the CMA memory anywhere, it throws an initialization error at the time of booting.

(We tested this version with the example hello_wifi, it seems that it does not take the wifi at first, but this is another topic).

Now we test the 1.17.1 release Release v1.17.1 · nerves-project/nerves_system_rpi4 · GitHub
It is clearly 64bit and reserves 256M of CMA without problem.

Then we test the 1.12.1 release Release v1.21.1 · nerves-project/nerves_system_rpi4 · GitHub
It reserves 320m of CMA without problem, which is typical for the raspbian image.

Later, based on your help, we upgraded version 1.19.0 to 32bits. This version is the last stable one with kernel 5.10 (even omx is supported). Here is the repository GitHub - joserpintuitivo/nerves_system_rpi4_gst_omx at 1.19.0-32

This version presents the same behavior cannot initialize the CMA.

We don’t know if there is some configuration in the kernel that we should take into account or another. But we are stuck on this issue.

If you can give us a second hand, it would be a great help for us and we would be grateful to you.

PS: I am attaching the log of version 1.19.0 to 32bits, where the error is shown.

Regards,

erlinit: The shell will be launched on tty 'tty1'.

erlinit: If you would like the shell to be on this tty,

erlinit: configure erlinit with '-c ttyS0'.

erlinit: The hostname is 'nerves-2cc8'.

[    4.029661] Out of memory: Killed process 81 (beam.smp) total-vm:146640kB, anon-rss:48320kB, file-rss:7444kB, shmem-rss:0kB, UID:0 pgtables:292kB oom_score_adj:0
[    4.063656] Out of memory: Killed process 124 (modprobe) total-vm:2244kB, anon-rss:36kB, file-rss:232kB, shmem-rss:0kB, UID:0 pgtables:32kB oom_score_adj:0
[    4.069312] [drm:vc4_bo_create [vc4]] *ERROR* Failed to allocate from CMA:
[    4.084924] vc4-drm gpu: [drm] *ERROR* fbdev: Failed to setup generic emulation (ret=-12)
[    4.098300] watchdog: watchdog0: watchdog did not stop!
[    5.278106] reboot: Restarting system
erlinit: The shell will be launched on tty 'tty1'.

erlinit: If you would like the shell to be on this tty,

erlinit: configure erlinit with '-c ttyS0'.

erlinit: The hostname is 'nerves-2cc8'.

[    4.091486] bcmgenet fd580000.ethernet eth0: failed to initialize Rx queues
[    4.099488] bcmgenet fd580000.ethernet eth0: failed to initialize DMA
[    5.429473] ieee80211 phy0: brcmf_cfg80211_attach: Failed to init iwm_priv (-12)
[    5.429485] ieee80211 phy0: brcmf_bus_started: failed: -12
[    5.429531] ieee80211 phy0: brcmf_attach: dongle is not responding: err=-12
[    5.489665] brcmfmac: brcmf_sdio_firmware_callback: brcmf_attach failed
[    5.557918] Out of memory: Killed process 81 (beam.smp) total-vm:144932kB, anon-rss:44948kB, file-rss:7444kB, shmem-rss:0kB, UID:0 pgtables:284kB oom_score_adj:0
[    5.598955] v3d fec00000.v3d: Failed to allocate page tables. Please ensure you have CMA enabled.
[    5.706547] 8<--- cut here ---
[    5.711742] Unable to handle kernel NULL pointer dereference at virtual address 00000000
[    5.722081] pgd = 22a48226
[    5.726896] [00000000] *pgd=80000000004003, *pmd=00000000
[    5.734444] Internal error: Oops: 80000207 [#1] PREEMPT SMP ARM
[    5.742469] Modules linked in: v3d gpu_sched dwc2 roles brcmfmac sha256_generic libsha256 cfg80211 brcmutil snd_bcm2835(C) bcm2835_isp(C) bcm2835_codec(C) videobuf2_dma_contig v4l2_mem2mem bcm2835_v4l2(C) bcm2835_mmal_vchiq(C) vc_sm_cma(C) videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common videodev mc rpivid_mem vc4 snd_soc_core snd_pcm_dmaengine snd_pcm snd_timer snd cec drm_kms_helper drm drm_panel_orientation_quirks syscopyarea sysfillrect sysimgblt fb_sys_fops uio_pdrv_genirq uio
[    5.793864] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G         C        5.10.88 #1
[    5.803479] Hardware name: BCM2711
[    5.808997] PC is at 0x0
[    5.813623] LR is at call_timer_fn+0x2c/0x224
[    5.820108] pc : [<00000000>]    lr : [<c02981a4>]    psr: 200f0113
[    5.828501] sp : c54d1e40  ip : 0000000b  fp : 00000000
[    5.835861] r10: ffff8d0a  r9 : 00000001  r8 : c54d1e74
[    5.843177] r7 : 00000101  r6 : 00000000  r5 : c54d0000  r4 : c0cf63c4
[    5.851835] r3 : c54d0000  r2 : ffff8d0a  r1 : 00000000  r0 : c0cf63c4
[    5.860436] Flags: nzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[    5.869688] Control: 30c5383d  Table: 05639700  DAC: 55555555
[    5.877535] Process swapper/2 (pid: 0, stack limit = 0x4e0bf1cb)
[    5.885607] Stack: (0xc54d1e40 to 0xc54d2000)
[    5.891991] 1e40: 00000001 c7fa0580 c54d0000 00000400 c7fa0580 c54d0000 c0cf63c4 00000000
[    5.902222] 1e60: c54d1e74 c02985b4 00000001 c0e03d00 c7fa1708 00000000 c7fa1758 c02a9618
[    5.912488] 1e80: 53bf1ce4 00000001 00000000 00000001 c7faa300 147ff8df c7fa168c c0c4d580
[    5.922744] 1ea0: c0e050ec c0e03d00 c54d0000 00000082 00000101 c0ac7e2c c54d1f30 c02986f4
[    5.932922] 1ec0: c0e03084 00000002 00000001 c0201234 c7fa17d0 c7fa17b0 c54d1ed0 c0e03080
[    5.943128] 1ee0: c0c492c8 c0c53840 0000000a c0c49258 c0afa2d0 ffff8d0b c0e03d00 00200042
[    5.953315] 1f00: c54d1f78 c0c53840 00000000 00000000 c0c53054 00000001 c54d1f78 c5415800
[    5.963498] 1f20: c54d1f30 c0220814 c0c53054 c0273c18 c54d1f78 c0e05454 c8802000 c0c53060
[    5.973630] 1f40: c880200c c54d1f58 c54d1f78 00000000 00000000 c0527fc8 c02079cc 600f0013
[    5.983771] 1f60: ffffffff c54d1fac 00007000 c54d0000 00000000 c0200af8 00000002 00000000
[    5.993877] 1f80: 00003bb8 c0215300 00000002 c54d0000 c0e04d1c c0e04d64 00007000 410fd083
[    6.003956] 1fa0: 00000000 00000000 00000001 c54d1fc8 c02079c8 c02079cc 600f0013 ffffffff
[    6.014019] 1fc0: c0e04d64 00000000 00000002 c086e144 00000002 c024ba00 00000092 00000000
[    6.024107] 1fe0: 30c0387d c54d1ff8 00007000 c024bd50 0548f9c0 002015f0 00000000 00000000
[    6.034200] [<c02981a4>] (call_timer_fn) from [<c02985b4>] (__run_timers.part.0+0x218/0x318)
[    6.044524] [<c02985b4>] (__run_timers.part.0) from [<c02986f4>] (run_timer_softirq+0x40/0x70)
[    6.055004] [<c02986f4>] (run_timer_softirq) from [<c0201234>] (__do_softirq+0x13c/0x454)
[    6.065003] [<c0201234>] (__do_softirq) from [<c0220814>] (irq_exit+0x108/0x10c)
[    6.074209] [<c0220814>] (irq_exit) from [<c0273c18>] (__handle_domain_irq+0xa8/0x110)
[    6.083868] [<c0273c18>] (__handle_domain_irq) from [<c0527fc8>] (gic_handle_irq+0x80/0x94)
[    6.094006] [<c0527fc8>] (gic_handle_irq) from [<c0200af8>] (__irq_svc+0x58/0x8c)
[    6.103208] Exception stack(0xc54d1f78 to 0xc54d1fc0)
[    6.109913] 1f60:                                                       00000002 00000000
[    6.119787] 1f80: 00003bb8 c0215300 00000002 c54d0000 c0e04d1c c0e04d64 00007000 410fd083
[    6.129648] 1fa0: 00000000 00000000 00000001 c54d1fc8 c02079c8 c02079cc 600f0013 ffffffff
[    6.139424] [<c0200af8>] (__irq_svc) from [<c02079cc>] (arch_cpu_idle+0x38/0x3c)
[    6.148441] [<c02079cc>] (arch_cpu_idle) from [<c086e144>] (default_idle_call+0x30/0x18c)
[    6.158258] [<c086e144>] (default_idle_call) from [<c024ba00>] (do_idle+0xc0/0x134)
[    6.167487] [<c024ba00>] (do_idle) from [<c024bd50>] (cpu_startup_entry+0x18/0x1c)
[    6.176634] [<c024bd50>] (cpu_startup_entry) from [<002015f0>] (0x2015f0)
[    6.184999] Code: bad PC value
[    6.189512] ---[ end trace 019c5b1e527b9fc1 ]---
[    6.204899] Kernel panic - not syncing: Fatal exception in interrupt
[    6.221808] Rebooting in 10 seconds..

fhunleth · December 20, 2022, 6:00pm

Ok. I have a long running process on my RPi4 now. When it completes and I get a moment, I’ll see if I can take a look.

In the meantime, here’s what I’d try:

Remove ramoops. I don’t think that it’s the issue, but it does a fixed memory address reservation that might interfere.
Manually mount the MicroSD card and copy all RaspberryPi .dtbo files to the overlays directory. Sometimes the RPi bootloader loads tries to load overlays and then falls back to ones it finds. I just don’t want that to be affecting anything, and I’ve run into issues there before.
Save build/linux-custom/.config to a file, perhaps, nerves-gst_omx.config. Then run make menuconfig to change the Linux config to use a built-in defconfig, "bcm2711", build Linux, the compare the resulting build/linux-custom/.config to nerves-gst_omx.config to see if any CMA-related options aren’t set.

I’m thinking that the 3rd option is probably really hard if you haven’t looked at Linux configs much since there will be tons of changes that aren’t relevant. Perhaps there’s a better way that I don’t know.

Frank

joserp93 · December 22, 2022, 6:05pm

Hi Frank, I tried different combinations, but none were successful.