Bug 13551

A very brief description...., please describe exactly what (and how) did you test, which result did you expect and which result did you get?

I suspect you did test something like the following:

  #include <QDebug>
  int main(int argc, char* argv[]) {
      qInfo() << "\u25C0";
      return 0;
  }

Please compile and run on the target and provide the output of the following call:

  $ ./test 2>&1 | hexdump -C

On my desktop system I get:

  00000000  e2 97 80 0a                                       |....|
  00000004

Which is the expected UTF-8 output for the Unicode/UTF-16 Black-Left-Pointing-Triangle ([1])...

[1] https://codepoints.net/U+25C0?lang=en

You test is very good to reproduce the probem in a simple way.

In my Ubuntu 20.04 I get the same result as you. It's correct.


But in the buildroot image for rpi3 (2021.02.rc2) I get:

  $ ./test 2>&1 | hexdump -C

  00000000  3f 0a                                             |?.|
  00000002


I don't understand why this is happening.

You test is very good to reproduce the probem in a simple way.

In my Ubuntu 20.04 I get the same result as you. It's correct.


But in the buildroot image for rpi3 (2021.02.rc2) I get:

  $ ./test 2>&1 | hexdump -C

  00000000  3f 0a                                             |?.|
  00000002


I don't understand why this is happening.

On RPi3:

  $ ./test 2>&1 | hexdump -C
  00000000  3f 0a                                             |?.|
  00000002

  $ LANG=en_US.utf8 ./test 2>&1 | hexdump -C
00000000  e2 97 80 0a                                       |....|
00000004

With the following defconfig:

  BR2_arm=y
  BR2_cortex_a53=y
  BR2_ARM_FPU_NEON_VFPV4=y
  BR2_TOOLCHAIN_BUILDROOT_GLIBC=y
  BR2_PACKAGE_GLIBC_UTILS=y
  BR2_BINUTILS_VERSION_2_36_X=y
  BR2_GCC_VERSION_10_X=y
  BR2_TOOLCHAIN_BUILDROOT_CXX=y
  BR2_PACKAGE_HOST_GDB=y
  BR2_GDB_VERSION_10=y
  BR2_ROOTFS_DEVICE_CREATION_DYNAMIC_EUDEV=y
  BR2_SYSTEM_DHCP="eth0"
  BR2_GENERATE_LOCALE="en_US"
  BR2_ROOTFS_POST_BUILD_SCRIPT="board/raspberrypi3/post-build.sh"
  BR2_ROOTFS_POST_IMAGE_SCRIPT="board/raspberrypi3/post-image.sh"
  BR2_ROOTFS_POST_SCRIPT_ARGS="--add-miniuart-bt-overlay"
  BR2_LINUX_KERNEL=y
  BR2_LINUX_KERNEL_CUSTOM_TARBALL=y
  BR2_LINUX_KERNEL_CUSTOM_TARBALL_LOCATION="$(call github,raspberrypi,linux,967d45b29ca2902f031b867809d72e3b3d623e7a)/linux-967d45b29ca2902f031b867809d72e3b3d623e7a.tar.gz"
  BR2_LINUX_KERNEL_DEFCONFIG="bcm2709"
  BR2_LINUX_KERNEL_DTS_SUPPORT=y
  BR2_LINUX_KERNEL_INTREE_DTS_NAME="bcm2710-rpi-3-b bcm2710-rpi-3-b-plus bcm2710-rpi-cm3"
  BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL=y
  BR2_PACKAGE_STRACE=y
  BR2_PACKAGE_QT5=y
  BR2_PACKAGE_QT5BASE_EXAMPLES=y
  BR2_PACKAGE_RPI_FIRMWARE=y
  BR2_PACKAGE_OPENSSH=y
  BR2_TARGET_ROOTFS_EXT2=y
  BR2_TARGET_ROOTFS_EXT2_4=y
  BR2_TARGET_ROOTFS_EXT2_SIZE="120M"
  # BR2_TARGET_ROOTFS_TAR is not set
  BR2_PACKAGE_HOST_DOSFSTOOLS=y
  BR2_PACKAGE_HOST_GENIMAGE=y
  BR2_PACKAGE_HOST_MTOOLS=y

So the issue is simply about the locale used. The default (C) locale doesn't handle unicode, but using an unicode locale fixes it.

Created attachment 8811 [details]
In QML UNICODE not work

Hello.

Yes, you are right when we use QT console.
But your proposed solution still does not work in QML. I attach a simple demo example.

In QML UNICODE not work.

(In reply to Alfredo Pons Menargues from comment #7)

Something went wrong with your attachment, no qml code/prject, only a text file duplicating your comment...

Created attachment 8816 [details]
Try now the attachment file

Sorry Peter,

I have attached it again.

(In reply to Alfredo Pons Menargues from comment #10)

Example works for me (Desktop/RPi4) as expected, both times the
'Trigram For Heaven' (U+2620, see [1]) is shown...

Back to the hint from my first reply: 

- please describe exactly what (and how) did you test
- please describe exactly which result did you expect and which result did you get?
- which target hardware, which buildroot version and provide your used
  defconfig


Test on RPi4 was done with the following defconfig:

BR2_aarch64=y
BR2_cortex_a72=y
BR2_ARM_FPU_VFPV4=y
BR2_ENABLE_DEBUG=y
BR2_OPTIMIZE_3=y
BR2_TOOLCHAIN_BUILDROOT_GLIBC=y
BR2_PACKAGE_HOST_LINUX_HEADERS_CUSTOM_5_10=y
BR2_BINUTILS_VERSION_2_36_X=y
BR2_GCC_VERSION_10_X=y
BR2_TOOLCHAIN_BUILDROOT_CXX=y
BR2_ROOTFS_DEVICE_CREATION_DYNAMIC_EUDEV=y
BR2_SYSTEM_DHCP="eth0"
BR2_GENERATE_LOCALE="en_US"
BR2_ROOTFS_POST_BUILD_SCRIPT="board/raspberrypi4-64/post-build.sh"
BR2_ROOTFS_POST_IMAGE_SCRIPT="board/raspberrypi4-64/post-image.sh"
BR2_ROOTFS_POST_SCRIPT_ARGS="--add-miniuart-bt-overlay --aarch64"
BR2_LINUX_KERNEL=y
BR2_LINUX_KERNEL_CUSTOM_TARBALL=y
BR2_LINUX_KERNEL_CUSTOM_TARBALL_LOCATION="$(call github,raspberrypi,linux,967d45b29ca2902f031b867809d72e3b3d623e7a)/linux-967d45b29ca2902f031b867809d72e3b3d623e7a.tar.gz"
BR2_LINUX_KERNEL_DEFCONFIG="bcm2711"
BR2_LINUX_KERNEL_DTS_SUPPORT=y
BR2_LINUX_KERNEL_INTREE_DTS_NAME="broadcom/bcm2711-rpi-4-b"
BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL=y
BR2_PACKAGE_STRACE=y
BR2_PACKAGE_DEJAVU=y
BR2_PACKAGE_MESA3D=y
BR2_PACKAGE_MESA3D_GALLIUM_DRIVER_KMSRO=y
BR2_PACKAGE_MESA3D_GALLIUM_DRIVER_V3D=y
BR2_PACKAGE_MESA3D_GALLIUM_DRIVER_VC4=y
BR2_PACKAGE_MESA3D_OPENGL_ES=y
BR2_PACKAGE_QT5=y
BR2_PACKAGE_QT5BASE_EXAMPLES=y
BR2_PACKAGE_QT5BASE_OPENGL_LIB=y
BR2_PACKAGE_QT5BASE_EGLFS=y
BR2_PACKAGE_QT5BASE_DEFAULT_QPA="eglfs"
BR2_PACKAGE_QT5BASE_FONTCONFIG=y
BR2_PACKAGE_QT5BASE_HARFBUZZ=y
BR2_PACKAGE_QT5BASE_GIF=y
BR2_PACKAGE_QT5BASE_JPEG=y
BR2_PACKAGE_QT5BASE_PNG=y
BR2_PACKAGE_QT5QUICKCONTROLS=y
BR2_PACKAGE_QT5QUICKCONTROLS2=y
BR2_PACKAGE_RPI_FIRMWARE=y
BR2_PACKAGE_RPI_FIRMWARE_VARIANT_PI4=y
BR2_PACKAGE_OPENSSH=y
BR2_TARGET_ROOTFS_EXT2=y
BR2_TARGET_ROOTFS_EXT2_4=y
BR2_TARGET_ROOTFS_EXT2_SIZE="384M"
# BR2_TARGET_ROOTFS_TAR is not set
BR2_PACKAGE_HOST_DOSFSTOOLS=y
BR2_PACKAGE_HOST_GENIMAGE=y
BR2_PACKAGE_HOST_MTOOLS=y

and 'dtoverlay=vc4-kms-v3d-pi4' enabled...

[1] https://codepoints.net/U+2630?lang=de

Created attachment 8821 [details]
My config file for buildroot-2021.02

Buildroot: buildroot-2021.02
CPU: eglfs/RPi3

In this scenario, if you run the tests of QT Console code:

  #include <QDebug>
  int main(int argc, char* argv[]) {
      qInfo() << "\u25C0";
      return 0;
  }

Run:

$ ./test 2>&1 | hexdump -C  --> NO UTF8


$ LANG=en_US.utf8 ./test 2>&1 | hexdump -C   --> UTF8 OK


In the console example the utf-8 text works if we pass in the environment the variable LANG=en_US.utf8, but in QML test this does not happen:


$ ./testQML --> NO UTF8

$ LANG=en_US.utf8 ./testQML --> NO UTF8

(In reply to Alfredo Pons Menargues from comment #12)

'NO UTF8' is again not very descriptive, a better/exact description would be
'I see only an square/replacement character' the default for an unknown character in your selected/installed font (Liberation)...., the solution for your problem
is to select/install an more complete font, e.g. DejaVu (BR2_PACKAGE_DEJAVU)...

Correct,
The solution for the problem
is to select/install an more complete font, e.g. DejaVu (BR2_PACKAGE_DEJAVU).