← Search

Table · dataset · 2026

<p>Five-run results on CMU-MOSI (mean sample SD).</p>

Listed in figshare

<div><p>Language use, vocal behavior, and facial activity provide complementary indicators of affective state, but multimodal models can be affected by noisy temporal observations and insufficient global guidance before cross-modal interaction.

Description

We developed a framework combining bidirectional gated recurrent unit encoders, attention pooling, dynamic hypernode injection, and graph attention fusion. Textual, acoustic, and visual sequences were mapped into a shared latent space and compressed into modality-level representations.

A sample-dependent hypernode and a learnable static prior were then injected through gated residual connections before graph propagation. The model was evaluated on CMU-MOSI and CMU-MOSEI using five random seeds and validation-MAE checkpoint selection. On CMU-MOSI, the model obtained an MAE of , a correlation of , and an Acc-7 of .

Read the rest (2 more)

On CMU-MOSEI, the corresponding results were , , and . A five-run comparison of pre-GAT, post-GAT, and no injection showed that pre-GAT injection provided the strongest overall regression-oriented balance, although post-GAT was slightly higher on MOSEI Acc-7. Gate, graph-attention, representation, and balanced case diagnostics showed that the injected prior was sample dependent and modality selective.

These findings support pre-propagation hypernode injection under complete, word-aligned tri-modal inputs, without establishing universal superiority or robustness to missing and corrupted modalities.</p></div>

Links

Where it is published

Catalogue records · 1

Topics

Provenance · 4 source records, 18 field assertions
SourceKeyLast seenRaw
HKU DataHub · removed upstreamoai:figshare.com:article/3401741613 d agoJSON v1
figshareoai:figshare.com:article/3401741613 d agoJSON v1
Loughborough Research Repository · removed upstreamoai:figshare.com:article/3401741612 d agoJSON v1
UP Research Data Repository · removed upstreamoai:figshare.com:article/3401741612 d agoJSON v1
FieldAssertionExtractorEvidence
access_levelsource · figshare comconnector:figshare_com@1.0.0
concepts[field].anzsrc:field:440710mapping · figshare comvocabulary-mapper@1.0.0keywords['Science Policy']
concepts[field].local:field:astronomymapping · figshare comconnector:figshare_com@1.0.0
concepts[field].local:field:chemistrymapping · figshare comconnector:figshare_com@1.0.0
concepts[field].local:field:computer-science-aimapping · figshare comconnector:figshare_com@1.0.0
concepts[field].local:field:earth-environmentalmapping · figshare comconnector:figshare_com@1.0.0
concepts[field].local:field:economics-financemapping · figshare comconnector:figshare_com@1.0.0
concepts[field].local:field:engineeringmapping · figshare comconnector:figshare_com@1.0.0
concepts[field].local:field:humanitiesmapping · figshare comconnector:figshare_com@1.0.0
concepts[field].local:field:life-sciencesmapping · figshare comconnector:figshare_com@1.0.0
concepts[field].local:field:medicine-healthmapping · figshare comconnector:figshare_com@1.0.0
concepts[field].local:field:ocean-atmosphericmapping · figshare comconnector:figshare_com@1.0.0
concepts[field].local:field:psychology-behavioralmapping · figshare comconnector:figshare_com@1.0.0
concepts[field].local:field:social-sciencemapping · figshare comconnector:figshare_com@1.0.0
descriptionsource · figshare comconnector:figshare_com@1.0.0/metadata/dc/description
licensesource · figshare comconnector:figshare_com@1.0.0/metadata/dc/rights
publication_datesource · figshare comconnector:figshare_com@1.0.0
titlesource · figshare comconnector:figshare_com@1.0.0/metadata/dc/title