xuning / sherpaonnx

转到一个项目
Toggle navigation Toggle navigation pinning
Fangjun Kuang
Committed by GitHub
315d8e2a 1 parent 109fb799

Publish `sherpa_onnx.har` for HarmonyOS (#1572)

内嵌 并排对比
正在显示 104 个修改的文件 包含 1888 行增加 和 9 行删除
name: har
on:
push:
branches:
- master
# - ohos-har
tags:
- 'v[0-9]+.[0-9]+.[0-9]+*'
workflow_dispatch:
concurrency:
group: har-${{ github.ref }}
cancel-in-progress: true
jobs:
har:
name: Har
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest]
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: ccache
uses: hendrikmuhs/ccache-action@v1.2
with:
key: har-linux
- name: cache-toolchain
id: cache-toolchain-ohos
uses: actions/cache@v4
with:
path: command-line-tools
key: commandline-tools-linux-x64-5.0.5.200.zip
- name: Download toolchain
if: steps.cache-toolchain-ohos.outputs.cache-hit != 'true'
shell: bash
run: |
curl -SL -O https://huggingface.co/csukuangfj/harmonyos-commandline-tools/resolve/main/commandline-tools-linux-x64-5.0.5.200.zip
unzip commandline-tools-linux-x64-5.0.5.200.zip
rm commandline-tools-linux-x64-5.0.5.200.zip
- name: Set environment variable
shell: bash
run: |
echo "$GITHUB_WORKSPACE/command-line-tools/sdk/default/openharmony/native/build-tools/cmake/bin" >> "$GITHUB_PATH"
which cmake
cmake --version
ls -lh $GITHUB_WORKSPACE/command-line-tools/sdk/default/openharmony/native/build/cmake/ohos.toolchain.cmake
echo "===="
cat $GITHUB_WORKSPACE/command-line-tools/sdk/default/openharmony/native/build/cmake/ohos.toolchain.cmake
echo "===="
# echo "$GITHUB_WORKSPACE/command-line-tools/sdk/default/openharmony/native/llvm/bin" >> "$GITHUB_PATH"
ls -lh $GITHUB_WORKSPACE/command-line-tools/sdk/default/openharmony/native/llvm/bin/
echo "--"
ls -lh $GITHUB_WORKSPACE/command-line-tools/sdk/default/openharmony/native/llvm/bin/*unknown*
cat $GITHUB_PATH
# /home/runner/work/onnxruntime-libs/onnxruntime-libs/command-line-tools/sdk/default/openharmony/native/llvm/bin/aarch64-unknown-linux-ohos-clang -v || true
export PATH=$PWD/command-line-tools/sdk/default/openharmony/native/llvm/bin:$PATH
echo "path: $PATH"
which aarch64-unknown-linux-ohos-clang++ || true
which aarch64-unknown-linux-ohos-clang || true
aarch64-unknown-linux-ohos-clang++ --version || true
aarch64-unknown-linux-ohos-clang --version || true
which armv7-unknown-linux-ohos-clang++
which armv7-unknown-linux-ohos-clang
armv7-unknown-linux-ohos-clang++ --version
armv7-unknown-linux-ohos-clang --version
which x86_64-unknown-linux-ohos-clang++
which x86_64-unknown-linux-ohos-clang
x86_64-unknown-linux-ohos-clang++ --version
x86_64-unknown-linux-ohos-clang --version
- name: Build libraries
shell: bash
run: |
export CMAKE_CXX_COMPILER_LAUNCHER=ccache
export PATH="/usr/lib/ccache:/usr/local/opt/ccache/libexec:$PATH"
cmake --version
export OHOS_SDK_NATIVE_DIR="$GITHUB_WORKSPACE/command-line-tools/sdk/default/openharmony/native"
./build-ohos-arm64-v8a.sh
./build-ohos-x86-64.sh
- name: Build Har
shell: bash
run: |
export PATH="$GITHUB_WORKSPACE/command-line-tools/bin:$PATH"
which hvigorw
pushd harmony-os/SherpaOnnxHar
hvigorw --mode module -p product=default -p module=sherpa_onnx@default assembleHar --analyze=normal --parallel --incremental --no-daemon
ls -lh ./sherpa_onnx/build/default/outputs/default/sherpa_onnx.har
cp -v ./sherpa_onnx/build/default/outputs/default/sherpa_onnx.har ../../
popd
ls -lh *.har
- name: Collect result
shell: bash
run: |
SHERPA_ONNX_VERSION=v$(grep "SHERPA_ONNX_VERSION" ./CMakeLists.txt | cut -d " " -f 2 | cut -d '"' -f 2)
echo "SHERPA_ONNX_VERSION=$SHERPA_ONNX_VERSION" >> "$GITHUB_ENV"
mv sherpa_onnx.har sherpa_onnx-$SHERPA_ONNX_VERSION.har
- uses: actions/upload-artifact@v4
with:
name: sherpa-onnx-har
path: ./sherpa_onnx*.har
- name: Release jar
if: (github.repository_owner == 'csukuangfj' || github.repository_owner == 'k2-fsa') && github.event_name == 'push' && contains(github.ref, 'refs/tags/')
uses: svenstaro/upload-release-action@v2
with:
file_glob: true
overwrite: true
file: ./*.har
# repo_name: k2-fsa/sherpa-onnx
# repo_token: ${{ secrets.UPLOAD_GH_SHERPA_ONNX_TOKEN }}
# tag: v1.10.32
- name: Publish to huggingface
if: (github.repository_owner == 'csukuangfj' || github.repository_owner == 'k2-fsa') && (github.event_name == 'push' || github.event_name == 'workflow_dispatch')
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
uses: nick-fields/retry@v3
with:
max_attempts: 20
timeout_seconds: 200
shell: bash
command: |
git config --global user.email "csukuangfj@gmail.com"
git config --global user.name "Fangjun Kuang"
rm -rf huggingface
export GIT_LFS_SKIP_SMUDGE=1
export GIT_CLONE_PROTECTION_ACTIVE=false
SHERPA_ONNX_VERSION=$(grep "SHERPA_ONNX_VERSION" ./CMakeLists.txt | cut -d " " -f 2 | cut -d '"' -f 2)
echo "SHERPA_ONNX_VERSION $SHERPA_ONNX_VERSION"
git clone https://csukuangfj:$HF_TOKEN@huggingface.co/csukuangfj/sherpa-onnx-harmony-os huggingface
cd huggingface
git fetch
git pull
git merge -m "merge remote" --ff origin main
d=har
mkdir -p $d
cp -v ../*.har $d/
git status
git lfs track "*.har"
git add .
git commit -m "add more hars"
git push https://csukuangfj:$HF_TOKEN@huggingface.co/csukuangfj/sherpa-onnx-harmony-os main
... ...
... ... @@ -18,14 +18,13 @@
### Supported platforms
|Architecture| Android | iOS | Windows | macOS | linux |
|------------|---------|---------|------------|-------|-------|
| x64 | ✔️ | | ✔️ | ✔️ | ✔️ |
| x86 | ✔️ | | ✔️ | | |
| arm64 | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ |
| arm32 | ✔️ | | | | ✔️ |
| riscv64 | | | | | ✔️ |
|Architecture| Android | iOS | Windows | macOS | linux | HarmonyOS |
|------------|---------|---------|------------|-------|-------|-----------|
| x64 | ✔️ | | ✔️ | ✔️ | ✔️ | ✔️ |
| x86 | ✔️ | | ✔️ | | | |
| arm64 | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ |
| arm32 | ✔️ | | | | ✔️ | ✔️ |
| riscv64 | | | | | ✔️ | |
### Supported programming languages
... ... @@ -65,6 +64,7 @@ on the following platforms and operating systems:
- Linux, macOS, Windows, openKylin
- Android, WearOS
- iOS
- HarmonyOS
- NodeJS
- WebAssembly
- [Raspberry Pi][Raspberry Pi]
... ...
... ... @@ -134,3 +134,9 @@ cp -fv $onnxruntime_dir/lib/libonnxruntime.so install/lib
rm -rf install/share
rm -rf install/lib/pkgconfig
d=../harmony-os/SherpaOnnxHar/sherpa_onnx/src/main/cpp/libs/arm64-v8a
if [ -d $d ]; then
cp -v install/lib/libsherpa-onnx-c-api.so $d/
cp -v install/lib/libonnxruntime.so $d/
fi
... ...
... ... @@ -134,3 +134,9 @@ cp -fv $onnxruntime_dir/lib/libonnxruntime.so install/lib
rm -rf install/share
rm -rf install/lib/pkgconfig
d=../harmony-os/SherpaOnnxHar/sherpa_onnx/src/main/cpp/libs/x86_64
if [ -d $d ]; then
cp -v install/lib/libsherpa-onnx-c-api.so $d/
cp -v install/lib/libonnxruntime.so $d/
fi
... ...
!build-profile.json5
... ...
/node_modules
/oh_modules
/local.properties
/.idea
**/build
/.hvigor
.cxx
/.clangd
/.clang-format
/.clang-tidy
**/.test
/.appanalyzer
\ No newline at end of file
... ...
{
"app": {
"bundleName": "com.k2fsa.sherpa.onnx",
"vendor": "example",
"versionCode": 1000000,
"versionName": "1.0.0",
"icon": "$media:app_icon",
"label": "$string:app_name"
}
}
... ...
{
"string": [
{
"name": "app_name",
"value": "SherpaOnnxHar"
}
]
}
... ...
# Introduction
How to build `sherpa_onnx.har` from the command line:
```bash
git clone https://github.com/k2-fsa/sherpa-onnx
cd sherpa-onnx
./build-ohos-arm64-v8a.sh
./build-ohos-x86-64.sh
cd harmony-os/SherpaOnnxHar
hvigorw clean --no-daemon
hvigorw --mode module -p product=default -p module=sherpa_onnx@default assembleHar --analyze=normal --parallel --incremental --no-daemon
ls -lh ./sherpa_onnx/build/default/outputs/default/sherpa_onnx.har
```
... ...
{
"app": {
"signingConfigs": [],
"products": [
{
"name": "default",
"signingConfig": "default",
"compatibleSdkVersion": "4.0.0(10)",
"runtimeOS": "HarmonyOS",
"buildOption": {
"strictMode": {
"caseSensitiveCheck": true,
}
}
}
],
"buildModeSet": [
{
"name": "debug",
},
{
"name": "release"
}
]
},
"modules": [
{
"name": "entry",
"srcPath": "./entry",
"targets": [
{
"name": "default",
"applyToProducts": [
"default"
]
}
]
},
{
"name": "sherpa_onnx",
"srcPath": "./sherpa_onnx",
}
]
}
\ No newline at end of file
... ...
{
"files": [
"**/*.ets"
],
"ignore": [
"**/src/ohosTest/**/*",
"**/src/test/**/*",
"**/src/mock/**/*",
"**/node_modules/**/*",
"**/oh_modules/**/*",
"**/build/**/*",
"**/.preview/**/*"
],
"ruleSet": [
"plugin:@performance/recommended",
"plugin:@typescript-eslint/recommended"
],
"rules": {
}
}
\ No newline at end of file
... ...
/node_modules
/oh_modules
/.preview
/build
/.cxx
/.test
\ No newline at end of file
... ...
{
"apiType": "stageMode",
"buildOption": {
},
"buildOptionSet": [
{
"name": "release",
"arkOptions": {
"obfuscation": {
"ruleOptions": {
"enable": false,
"files": [
"./obfuscation-rules.txt"
]
}
}
}
},
],
"targets": [
{
"name": "default"
},
{
"name": "ohosTest",
}
]
}
\ No newline at end of file
... ...
import { hapTasks } from '@ohos/hvigor-ohos-plugin';
export default {
system: hapTasks, /* Built-in plugin of Hvigor. It cannot be modified. */
plugins:[] /* Custom plugin to extend the functionality of Hvigor. */
}
... ...
# Define project specific obfuscation rules here.
# You can include the obfuscation configuration files in the current module's build-profile.json5.
#
# For more details, see
# https://developer.huawei.com/consumer/cn/doc/harmonyos-guides-V5/source-obfuscation-V5
# Obfuscation options:
# -disable-obfuscation: disable all obfuscations
# -enable-property-obfuscation: obfuscate the property names
# -enable-toplevel-obfuscation: obfuscate the names in the global scope
# -compact: remove unnecessary blank spaces and all line feeds
# -remove-log: remove all console.* statements
# -print-namecache: print the name cache that contains the mapping from the old names to new names
# -apply-namecache: reuse the given cache file
# Keep options:
# -keep-property-name: specifies property names that you want to keep
# -keep-global-name: specifies names that you want to keep in the global scope
-enable-property-obfuscation
-enable-toplevel-obfuscation
-enable-filename-obfuscation
-enable-export-obfuscation
\ No newline at end of file
... ...
{
"name": "entry",
"version": "1.0.0",
"description": "Please describe the basic information.",
"main": "",
"author": "",
"license": "",
"dependencies": {}
}
... ...
import AbilityConstant from '@ohos.app.ability.AbilityConstant';
import hilog from '@ohos.hilog';
import UIAbility from '@ohos.app.ability.UIAbility';
import Want from '@ohos.app.ability.Want';
import window from '@ohos.window';
export default class EntryAbility extends UIAbility {
onCreate(want: Want, launchParam: AbilityConstant.LaunchParam): void {
hilog.info(0x0000, 'testTag', '%{public}s', 'Ability onCreate');
}
onDestroy(): void {
hilog.info(0x0000, 'testTag', '%{public}s', 'Ability onDestroy');
}
onWindowStageCreate(windowStage: window.WindowStage): void {
// Main window is created, set main page for this ability
hilog.info(0x0000, 'testTag', '%{public}s', 'Ability onWindowStageCreate');
windowStage.loadContent('pages/Index', (err) => {
if (err.code) {
hilog.error(0x0000, 'testTag', 'Failed to load the content. Cause: %{public}s', JSON.stringify(err) ?? '');
return;
}
hilog.info(0x0000, 'testTag', 'Succeeded in loading the content.');
});
}
onWindowStageDestroy(): void {
// Main window is destroyed, release UI related resources
hilog.info(0x0000, 'testTag', '%{public}s', 'Ability onWindowStageDestroy');
}
onForeground(): void {
// Ability has brought to foreground
hilog.info(0x0000, 'testTag', '%{public}s', 'Ability onForeground');
}
onBackground(): void {
// Ability has back to background
hilog.info(0x0000, 'testTag', '%{public}s', 'Ability onBackground');
}
}
... ...
import hilog from '@ohos.hilog';
import BackupExtensionAbility, { BundleVersion } from '@ohos.application.BackupExtensionAbility';
export default class EntryBackupAbility extends BackupExtensionAbility {
async onBackup() {
hilog.info(0x0000, 'testTag', 'onBackup ok');
}
async onRestore(bundleVersion: BundleVersion) {
hilog.info(0x0000, 'testTag', 'onRestore ok %{public}s', JSON.stringify(bundleVersion));
}
}
\ No newline at end of file
... ...
@Entry
@Component
struct Index {
@State message: string = 'Hello World';
build() {
Row() {
Column() {
Text(this.message)
.fontSize(50)
.fontWeight(FontWeight.Bold)
}
.width('100%')
}
.height('100%')
}
}
\ No newline at end of file
... ...
{
"module": {
"name": "entry",
"type": "entry",
"description": "$string:module_desc",
"mainElement": "EntryAbility",
"deviceTypes": [
"phone",
"tablet",
"2in1"
],
"deliveryWithInstall": true,
"installationFree": false,
"pages": "$profile:main_pages",
"abilities": [
{
"name": "EntryAbility",
"srcEntry": "./ets/entryability/EntryAbility.ets",
"description": "$string:EntryAbility_desc",
"icon": "$media:layered_image",
"label": "$string:EntryAbility_label",
"startWindowIcon": "$media:startIcon",
"startWindowBackground": "$color:start_window_background",
"exported": true,
"skills": [
{
"entities": [
"entity.system.home"
],
"actions": [
"action.system.home"
]
}
]
}
],
"extensionAbilities": [
{
"name": "EntryBackupAbility",
"srcEntry": "./ets/entrybackupability/EntryBackupAbility.ets",
"type": "backup",
"exported": false,
"metadata": [
{
"name": "ohos.extension.backup",
"resource": "$profile:backup_config"
}
],
}
]
}
}
\ No newline at end of file
... ...
{
"color": [
{
"name": "start_window_background",
"value": "#FFFFFF"
}
]
}
\ No newline at end of file
... ...
{
"string": [
{
"name": "module_desc",
"value": "module description"
},
{
"name": "EntryAbility_desc",
"value": "description"
},
{
"name": "EntryAbility_label",
"value": "label"
}
]
}
\ No newline at end of file
... ...
{
"layered-image":
{
"background" : "$media:background",
"foreground" : "$media:foreground"
}
}
\ No newline at end of file
... ...
{
"allowToBackupRestore": true
}
\ No newline at end of file
... ...
{
"string": [
{
"name": "module_desc",
"value": "module description"
},
{
"name": "EntryAbility_desc",
"value": "description"
},
{
"name": "EntryAbility_label",
"value": "label"
}
]
}
\ No newline at end of file
... ...
{
"string": [
{
"name": "module_desc",
"value": "模块描述"
},
{
"name": "EntryAbility_desc",
"value": "description"
},
{
"name": "EntryAbility_label",
"value": "label"
}
]
}
\ No newline at end of file
... ...
import hilog from '@ohos.hilog';
import { describe, beforeAll, beforeEach, afterEach, afterAll, it, expect } from '@ohos/hypium';
export default function abilityTest() {
describe('ActsAbilityTest', () => {
// Defines a test suite. Two parameters are supported: test suite name and test suite function.
beforeAll(() => {
// Presets an action, which is performed only once before all test cases of the test suite start.
// This API supports only one parameter: preset action function.
})
beforeEach(() => {
// Presets an action, which is performed before each unit test case starts.
// The number of execution times is the same as the number of test cases defined by **it**.
// This API supports only one parameter: preset action function.
})
afterEach(() => {
// Presets a clear action, which is performed after each unit test case ends.
// The number of execution times is the same as the number of test cases defined by **it**.
// This API supports only one parameter: clear action function.
})
afterAll(() => {
// Presets a clear action, which is performed after all test cases of the test suite end.
// This API supports only one parameter: clear action function.
})
it('assertContain', 0, () => {
// Defines a test case. This API supports three parameters: test case name, filter parameter, and test case function.
hilog.info(0x0000, 'testTag', '%{public}s', 'it begin');
let a = 'abc';
let b = 'b';
// Defines a variety of assertion methods, which are used to declare expected boolean conditions.
expect(a).assertContain(b);
expect(a).assertEqual(a);
})
})
}
\ No newline at end of file
... ...
import abilityTest from './Ability.test';
export default function testsuite() {
abilityTest();
}
\ No newline at end of file
... ...
{
"module": {
"name": "entry_test",
"type": "feature",
"deviceTypes": [
"phone",
"tablet",
"2in1"
],
"deliveryWithInstall": true,
"installationFree": false
}
}
... ...
import localUnitTest from './LocalUnit.test';
export default function testsuite() {
localUnitTest();
}
\ No newline at end of file
... ...
import { describe, beforeAll, beforeEach, afterEach, afterAll, it, expect } from '@ohos/hypium';
export default function localUnitTest() {
describe('localUnitTest', () => {
// Defines a test suite. Two parameters are supported: test suite name and test suite function.
beforeAll(() => {
// Presets an action, which is performed only once before all test cases of the test suite start.
// This API supports only one parameter: preset action function.
});
beforeEach(() => {
// Presets an action, which is performed before each unit test case starts.
// The number of execution times is the same as the number of test cases defined by **it**.
// This API supports only one parameter: preset action function.
});
afterEach(() => {
// Presets a clear action, which is performed after each unit test case ends.
// The number of execution times is the same as the number of test cases defined by **it**.
// This API supports only one parameter: clear action function.
});
afterAll(() => {
// Presets a clear action, which is performed after all test cases of the test suite end.
// This API supports only one parameter: clear action function.
});
it('assertContain', 0, () => {
// Defines a test case. This API supports three parameters: test case name, filter parameter, and test case function.
let a = 'abc';
let b = 'b';
// Defines a variety of assertion methods, which are used to declare expected boolean conditions.
expect(a).assertContain(b);
expect(a).assertEqual(a);
});
});
}
\ No newline at end of file
... ...
{
"modelVersion": "5.0.0",
"dependencies": {
},
"execution": {
// "analyze": "normal", /* Define the build analyze mode. Value: [ "normal" | "advanced" | false ]. Default: "normal" */
// "daemon": true, /* Enable daemon compilation. Value: [ true | false ]. Default: true */
// "incremental": true, /* Enable incremental compilation. Value: [ true | false ]. Default: true */
// "parallel": true, /* Enable parallel compilation. Value: [ true | false ]. Default: true */
// "typeCheck": false, /* Enable typeCheck. Value: [ true | false ]. Default: false */
},
"logging": {
// "level": "info" /* Define the log level. Value: [ "debug" | "info" | "warn" | "error" ]. Default: "info" */
},
"debugging": {
// "stacktrace": false /* Disable stacktrace compilation. Value: [ true | false ]. Default: false */
},
"nodeOptions": {
// "maxOldSpaceSize": 8192 /* Enable nodeOptions maxOldSpaceSize compilation. Unit M. Used for the daemon process. Default: 8192*/
// "exposeGC": true /* Enable to trigger garbage collection explicitly. Default: true*/
}
}
... ...
import { appTasks } from '@ohos/hvigor-ohos-plugin';
export default {
system: appTasks, /* Built-in plugin of Hvigor. It cannot be modified. */
plugins:[] /* Custom plugin to extend the functionality of Hvigor. */
}
... ...
# Notes
## How to publish a package
Please see
- <https://ohpm.openharmony.cn/#/cn/help/publishrequirefile>
- <https://ohpm.openharmony.cn/#/cn/help/createandpublish>
- <https://developer.huawei.com/consumer/cn/doc/harmonyos-guides-V5/ide-har-publish-V5>
## How to sign the HAP file from commandline
Please see
<https://developer.huawei.com/consumer/cn/doc/harmonyos-guides-V5/ide-command-line-building-app-V5>
... ...
{
"meta": {
"stableOrder": true
},
"lockfileVersion": 3,
"ATTENTION": "THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.",
"specifiers": {
"@ohos/hypium@1.0.19": "@ohos/hypium@1.0.19"
},
"packages": {
"@ohos/hypium@1.0.19": {
"name": "@ohos/hypium",
"version": "1.0.19",
"integrity": "sha512-cEjDgLFCm3cWZDeRXk7agBUkPqjWxUo6AQeiu0gEkb3J8ESqlduQLSIXeo3cCsm8U/asL7iKjF85ZyOuufAGSQ==",
"resolved": "https://ohpm.openharmony.cn/ohpm/@ohos/hypium/-/hypium-1.0.19.har",
"registryType": "ohpm"
}
}
}
\ No newline at end of file
... ...
{
"modelVersion": "5.0.0",
"description": "Please describe the basic information.",
"dependencies": {
},
"devDependencies": {
"@ohos/hypium": "1.0.19"
}
}
... ...
/node_modules
/oh_modules
/.preview
/build
/.cxx
/.test
\ No newline at end of file
... ...
/**
* Use these variables when you tailor your ArkTS code. They must be of the const type.
*/
export const HAR_VERSION = '1.0.0';
export const BUILD_MODE_NAME = 'debug';
export const DEBUG = true;
export const TARGET_NAME = 'default';
/**
* BuildProfile Class is used only for compatibility purposes.
*/
export default class BuildProfile {
static readonly HAR_VERSION = HAR_VERSION;
static readonly BUILD_MODE_NAME = BUILD_MODE_NAME;
static readonly DEBUG = DEBUG;
static readonly TARGET_NAME = TARGET_NAME;
}
\ No newline at end of file
... ...
export { readWave, readWaveFromBinary } from "libsherpa_onnx.so";
export {
CircularBuffer,
SileroVadConfig,
SpeechSegment,
Vad,
VadConfig,
} from './src/main/ets/components/Vad';
export {
Samples,
OfflineStream,
FeatureConfig,
OfflineTransducerModelConfig,
OfflineParaformerModelConfig,
OfflineNemoEncDecCtcModelConfig,
OfflineWhisperModelConfig,
OfflineTdnnModelConfig,
OfflineSenseVoiceModelConfig,
OfflineMoonshineModelConfig,
OfflineModelConfig,
OfflineLMConfig,
OfflineRecognizerConfig,
OfflineRecognizerResult,
OfflineRecognizer,
} from './src/main/ets/components/NonStreamingAsr';
export {
OnlineStream,
OnlineTransducerModelConfig,
OnlineParaformerModelConfig,
OnlineZipformer2CtcModelConfig,
OnlineModelConfig,
OnlineCtcFstDecoderConfig,
OnlineRecognizerConfig,
OnlineRecognizerResult,
OnlineRecognizer,
} from './src/main/ets/components/StreamingAsr';
... ...
# Introduction
[sherpa-onnx](https://github.com/k2-fsa/sherpa-onnx) is one of the deployment
frameworks of [Next-gen Kaldi](https://github.com/k2-fsa).
It supports speech-to-text, text-to-speech, speaker diarization, and VAD using
onnxruntime without Internet connection.
It also supports embedded systems, Android, iOS, HarmonyOS,
Raspberry Pi, RISC-V, x86_64 servers, websocket server/client,
C/C++, Python, Kotlin, C#, Go, NodeJS, Java, Swift, Dart, JavaScript,
Flutter, Object Pascal, Lazarus, Rust, etc.
... ...
{
"apiType": "stageMode",
"buildOption": {
"externalNativeOptions": {
"path": "./src/main/cpp/CMakeLists.txt",
"arguments": "",
"cppFlags": "",
"abiFilters": [
"arm64-v8a",
"x86_64",
],
},
},
"buildOptionSet": [
{
"name": "release",
"arkOptions": {
"obfuscation": {
"ruleOptions": {
"enable": false,
"files": [
"./obfuscation-rules.txt"
]
},
"consumerFiles": [
"./consumer-rules.txt"
]
}
},
"nativeLib": {
"debugSymbol": {
"strip": true,
"exclude": []
}
}
},
],
"targets": [
{
"name": "default"
},
{
"name": "ohosTest"
}
]
}
... ...
import { harTasks } from '@ohos/hvigor-ohos-plugin';
export default {
system: harTasks, /* Built-in plugin of Hvigor. It cannot be modified. */
plugins:[] /* Custom plugin to extend the functionality of Hvigor. */
}
... ...
# Define project specific obfuscation rules here.
# You can include the obfuscation configuration files in the current module's build-profile.json5.
#
# For more details, see
# https://developer.huawei.com/consumer/cn/doc/harmonyos-guides-V5/source-obfuscation-V5
# Obfuscation options:
# -disable-obfuscation: disable all obfuscations
# -enable-property-obfuscation: obfuscate the property names
# -enable-toplevel-obfuscation: obfuscate the names in the global scope
# -compact: remove unnecessary blank spaces and all line feeds
# -remove-log: remove all console.* statements
# -print-namecache: print the name cache that contains the mapping from the old names to new names
# -apply-namecache: reuse the given cache file
# Keep options:
# -keep-property-name: specifies property names that you want to keep
# -keep-global-name: specifies names that you want to keep in the global scope
-enable-property-obfuscation
-enable-toplevel-obfuscation
-enable-filename-obfuscation
-enable-export-obfuscation
\ No newline at end of file
... ...
{
"meta": {
"stableOrder": true
},
"lockfileVersion": 3,
"ATTENTION": "THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.",
"specifiers": {
"libsherpa_onnx.so@src/main/cpp/types/libsherpa_onnx": "libsherpa_onnx.so@src/main/cpp/types/libsherpa_onnx"
},
"packages": {
"libsherpa_onnx.so@src/main/cpp/types/libsherpa_onnx": {
"name": "libsherpa_onnx.so",
"version": "1.0.0",
"resolved": "src/main/cpp/types/libsherpa_onnx",
"registryType": "local"
}
}
}
\ No newline at end of file
... ...
{
"name": "sherpa_onnx",
"version": "1.0.0",
"description": "Speech-to-text, text-to-speech, and speaker diarization using Next-gen Kaldi without internet connection",
"main": "Index.ets",
"author": "The next-gen Kaldi team",
"license": "Apache-2.0",
"homepage": "https://github.com/k2-fsa/sherpa-onnx",
"repository": "https://github.com/k2-fsa/sherpa-onnx/tree/master/harmonyos-SherpaOnnxHar",
"dependencies": {
"libsherpa_onnx.so": "file:./src/main/cpp/types/libsherpa_onnx"
},
"keywords": [
"tts",
"asr",
"locally",
"diarization",
"privacy",
"open-source",
"speaker",
],
"bugs": {
"url": "https://github.com/k2-fsa/sherpa-onnx/issues"
},
}
... ...
# the minimum version of CMake.
cmake_minimum_required(VERSION 3.13.0)
project(myNpmLib)
# Disable warning about
#
# "The DOWNLOAD_EXTRACT_TIMESTAMP option was not given and policy CMP0135 is
# not set.
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.24.0")
cmake_policy(SET CMP0135 NEW)
endif()
set(NATIVERENDER_ROOT_PATH ${CMAKE_CURRENT_SOURCE_DIR})
if(DEFINED PACKAGE_FIND_FILE)
include(${PACKAGE_FIND_FILE})
endif()
include_directories(${NATIVERENDER_ROOT_PATH}
${NATIVERENDER_ROOT_PATH}/include)
include(FetchContent)
FetchContent_Declare(node_addon_api
GIT_REPOSITORY "https://github.com/nodejs/node-addon-api.git"
GIT_TAG c679f6f4c9dc6bf9fc0d99cbe5982bd24a5e2c7b
PATCH_COMMAND git checkout . && git apply --ignore-whitespace "${CMAKE_CURRENT_LIST_DIR}/my-patch.diff"
)
FetchContent_MakeAvailable(node_addon_api)
FetchContent_GetProperties(node_addon_api)
if(NOT node_addon_api_POPULATED)
message(STATUS "Downloading node-addon-api from")
FetchContent_Populate(node_addon_api)
endif()
message(STATUS "node-addon-api is downloaded to ${node_addon_api_SOURCE_DIR}")
include_directories(${node_addon_api_SOURCE_DIR})
add_library(sherpa_onnx SHARED
audio-tagging.cc
keyword-spotting.cc
non-streaming-asr.cc
non-streaming-speaker-diarization.cc
non-streaming-tts.cc
punctuation.cc
sherpa-onnx-node-addon-api.cc
speaker-identification.cc
spoken-language-identification.cc
streaming-asr.cc
vad.cc
wave-reader.cc
wave-writer.cc
)
add_library(sherpa_onnx_c_api SHARED IMPORTED)
set_target_properties(sherpa_onnx_c_api
PROPERTIES
IMPORTED_LOCATION ${CMAKE_CURRENT_SOURCE_DIR}/libs/${OHOS_ARCH}/libsherpa-onnx-c-api.so)
add_library(onnxruntime SHARED IMPORTED)
set_target_properties(onnxruntime
PROPERTIES
IMPORTED_LOCATION ${CMAKE_CURRENT_SOURCE_DIR}/libs/${OHOS_ARCH}/libonnxruntime.so)
target_link_libraries(sherpa_onnx PUBLIC libace_napi.z.so
libhilog_ndk.z.so # for hilog
librawfile.z.so
sherpa_onnx_c_api onnxruntime
)
... ...
# Node
[./c-api.h](./c-api.h) is a symbolic link to
https://github.com/k2-fsa/sherpa-onnx/blob/master/sherpa-onnx/c-api/c-api.h
If you are using Windows, then you need to manually replace this file with
https://github.com/k2-fsa/sherpa-onnx/blob/master/sherpa-onnx/c-api/c-api.h
since Windows does not support symbolic links.
... ...
../../../../../../../../../sherpa-onnx/c-api/c-api.h
\ No newline at end of file
... ...
# Introduction
You need to get the following four `.so` files using
- [build-ohos-arm64-v8a.sh](https://github.com/k2-fsa/sherpa-onnx/blob/master/build-ohos-arm64-v8a.sh)
- [build-ohos-x86-64.sh](https://github.com/k2-fsa/sherpa-onnx/blob/master/build-ohos-x86-64.sh)
```
.
├── README.md
├── arm64-v8a
│   ├── libonnxruntime.so
│   └── libsherpa-onnx-c-api.so
└── x86_64
├── libonnxruntime.so
└── libsherpa-onnx-c-api.so
```
... ...
... ... @@ -7,6 +7,18 @@
#include <algorithm>
#include <string>
#if __OHOS__
#include "rawfile/raw_file_manager.h"
#include "hilog/log.h"
#undef LOG_DOMAIN
#undef LOG_TAG
// https://gitee.com/openharmony/docs/blob/145a084f0b742e4325915e32f8184817927d1251/en/contribute/OpenHarmony-Log-guide.md#hilog-api-usage-specifications
#define LOG_DOMAIN 0x6666
#define LOG_TAG "sherpa_onnx"
#endif
#define SHERPA_ONNX_ASSIGN_ATTR_STR(c_name, js_name) \
do { \
if (o.Has(#js_name) && o.Get(#js_name).IsString()) { \
... ...
diff --git a/napi-inl.h b/napi-inl.h
index e7141c0..0fd90d8 100644
--- a/napi-inl.h
+++ b/napi-inl.h
@@ -2156,7 +2156,8 @@ inline ArrayBuffer::ArrayBuffer(napi_env env, napi_value value)
inline void* ArrayBuffer::Data() {
void* data;
- napi_status status = napi_get_arraybuffer_info(_env, _value, &data, nullptr);
+ size_t byte_length;
+ napi_status status = napi_get_arraybuffer_info(_env, _value, &data, &byte_length);
NAPI_THROW_IF_FAILED(_env, status, nullptr);
return data;
}
... ...
... ... @@ -191,6 +191,17 @@ static SherpaOnnxOfflineLMConfig GetOfflineLMConfig(Napi::Object obj) {
static Napi::External<SherpaOnnxOfflineRecognizer>
CreateOfflineRecognizerWrapper(const Napi::CallbackInfo &info) {
Napi::Env env = info.Env();
#if __OHOS__
// the last argument is the NativeResourceManager
if (info.Length() != 2) {
std::ostringstream os;
os << "Expect only 2 arguments. Given: " << info.Length();
Napi::TypeError::New(env, os.str()).ThrowAsJavaScriptException();
return {};
}
#else
if (info.Length() != 1) {
std::ostringstream os;
os << "Expect only 1 argument. Given: " << info.Length();
... ... @@ -199,6 +210,7 @@ CreateOfflineRecognizerWrapper(const Napi::CallbackInfo &info) {
return {};
}
#endif
if (!info[0].IsObject()) {
Napi::TypeError::New(env, "Expect an object as the argument")
... ... @@ -223,8 +235,15 @@ CreateOfflineRecognizerWrapper(const Napi::CallbackInfo &info) {
SHERPA_ONNX_ASSIGN_ATTR_STR(rule_fars, ruleFars);
SHERPA_ONNX_ASSIGN_ATTR_FLOAT(blank_penalty, blankPenalty);
#if __OHOS__
std::unique_ptr<NativeResourceManager, decltype(&OH_ResourceManager_ReleaseNativeResourceManager)> mgr (OH_ResourceManager_InitNativeResourceManager(env, info[1]), &OH_ResourceManager_ReleaseNativeResourceManager);
const SherpaOnnxOfflineRecognizer *recognizer =
SherpaOnnxCreateOfflineRecognizerOHOS(&c, mgr.get());
#else
const SherpaOnnxOfflineRecognizer *recognizer =
SherpaOnnxCreateOfflineRecognizer(&c);
#endif
SHERPA_ONNX_DELETE_C_STR(c.model_config.transducer.encoder);
SHERPA_ONNX_DELETE_C_STR(c.model_config.transducer.decoder);
... ... @@ -374,8 +393,15 @@ static void AcceptWaveformOfflineWrapper(const Napi::CallbackInfo &info) {
Napi::Float32Array samples = obj.Get("samples").As<Napi::Float32Array>();
int32_t sample_rate = obj.Get("sampleRate").As<Napi::Number>().Int32Value();
#if __OHOS__
// Note(fangjun): For unknown reasons on HarmonyOS, we need to divide it by
// sizeof(float) here
SherpaOnnxAcceptWaveformOffline(stream, sample_rate, samples.Data(),
samples.ElementLength() / sizeof(float));
#else
SherpaOnnxAcceptWaveformOffline(stream, sample_rate, samples.Data(),
samples.ElementLength());
#endif
}
static void DecodeOfflineStreamWrapper(const Napi::CallbackInfo &info) {
... ...
... ... @@ -147,6 +147,16 @@ static SherpaOnnxOnlineCtcFstDecoderConfig GetCtcFstDecoderConfig(
static Napi::External<SherpaOnnxOnlineRecognizer> CreateOnlineRecognizerWrapper(
const Napi::CallbackInfo &info) {
Napi::Env env = info.Env();
#if __OHOS__
if (info.Length() != 2) {
std::ostringstream os;
os << "Expect only 2 arguments. Given: " << info.Length();
Napi::TypeError::New(env, os.str()).ThrowAsJavaScriptException();
return {};
}
#else
if (info.Length() != 1) {
std::ostringstream os;
os << "Expect only 1 argument. Given: " << info.Length();
... ... @@ -155,6 +165,7 @@ static Napi::External<SherpaOnnxOnlineRecognizer> CreateOnlineRecognizerWrapper(
return {};
}
#endif
if (!info[0].IsObject()) {
Napi::TypeError::New(env, "Expect an object as the argument")
... ... @@ -199,8 +210,15 @@ static Napi::External<SherpaOnnxOnlineRecognizer> CreateOnlineRecognizerWrapper(
c.ctc_fst_decoder_config = GetCtcFstDecoderConfig(o);
#if __OHOS__
std::unique_ptr<NativeResourceManager, decltype(&OH_ResourceManager_ReleaseNativeResourceManager)> mgr (OH_ResourceManager_InitNativeResourceManager(env, info[1]), &OH_ResourceManager_ReleaseNativeResourceManager);
const SherpaOnnxOnlineRecognizer *recognizer =
SherpaOnnxCreateOnlineRecognizerOHOS(&c, mgr.get());
#else
const SherpaOnnxOnlineRecognizer *recognizer =
SherpaOnnxCreateOnlineRecognizer(&c);
#endif
if (c.model_config.transducer.encoder) {
delete[] c.model_config.transducer.encoder;
... ... @@ -385,8 +403,13 @@ static void AcceptWaveformWrapper(const Napi::CallbackInfo &info) {
Napi::Float32Array samples = obj.Get("samples").As<Napi::Float32Array>();
int32_t sample_rate = obj.Get("sampleRate").As<Napi::Number>().Int32Value();
#if __OHOS__
SherpaOnnxOnlineStreamAcceptWaveform(stream, sample_rate, samples.Data(),
samples.ElementLength() / sizeof(float));
#else
SherpaOnnxOnlineStreamAcceptWaveform(stream, sample_rate, samples.Data(),
samples.ElementLength());
#endif
}
static Napi::Boolean IsOnlineStreamReadyWrapper(
... ...
export const readWave: (filename: string, enableExternalBuffer: boolean = true) => {samples: Float32Array, sampleRate: number};
export const readWaveFromBinary: (data: Uint8Array, enableExternalBuffer: boolean = true) => {samples: Float32Array, sampleRate: number};
export const createCircularBuffer: (capacity: number) => object;
export const circularBufferPush: (handle: object, samples: Float32Array) => void;
export const circularBufferGet: (handle: object, index: number, n: number, enableExternalBuffer: boolean = true) => Float32Array;
export const circularBufferPop: (handle: object, n: number) => void;
export const circularBufferSize: (handle: object) => number;
export const circularBufferHead: (handle: object) => number;
export const circularBufferReset: (handle: object) => void;
export const createVoiceActivityDetector: (config: object, bufferSizeInSeconds: number, mgr?: object) => object;
export const voiceActivityDetectorAcceptWaveform: (handle: object, samples: Float32Array) => void;
export const voiceActivityDetectorIsEmpty: (handle: object) => boolean;
export const voiceActivityDetectorIsDetected: (handle: object) => boolean;
export const voiceActivityDetectorPop: (handle: object) => void;
export const voiceActivityDetectorClear: (handle: object) => void;
export const voiceActivityDetectorFront: (handle: object, enableExternalBuffer: boolean = true) => {samples: Float32Array, start: number};
export const voiceActivityDetectorReset: (handle: object) => void;
export const voiceActivityDetectorFlush: (handle: object) => void;
export const createOfflineRecognizer: (config: object, mgr?: object) => object;
export const createOfflineStream: (handle: object) => object;
export const acceptWaveformOffline: (handle: object, audio: object) => void;
export const decodeOfflineStream: (handle: object, streamHandle: object) => void;
export const getOfflineStreamResultAsJson: (streamHandle: object) => string;
export const createOnlineRecognizer: (config: object, mgr?: object) => object;
export const createOnlineStream: (handle: object) => object;
export const acceptWaveformOnline: (handle: object, audio: object) => void;
export const inputFinished: (streamHandle: object) => void;
export const isOnlineStreamReady: (handle: object, streamHandle: object) => boolean;
export const decodeOnlineStream: (handle: object, streamHandle: object) => void;
export const isEndpoint: (handle: object, streamHandle: object) => boolean;
export const reset: (handle: object, streamHandle: object) => void;
export const getOnlineStreamResultAsJson: (handle: object, streamHandle: object) => string;
... ...
{
"name": "libsherpa_onnx.so",
"types": "./Index.d.ts",
"version": "1.0.0",
"description": "Please describe the basic information."
}
\ No newline at end of file
... ...
... ... @@ -67,7 +67,14 @@ static void CircularBufferPushWrapper(const Napi::CallbackInfo &info) {
}
Napi::Float32Array data = info[1].As<Napi::Float32Array>();
#if __OHOS__
// Note(fangjun): Normally, we don't need to divied it by sizeof(float).
// However, data.ElementLength() here returns number of bytes, not number of elements.
SherpaOnnxCircularBufferPush(buf, data.Data(), data.ElementLength() / sizeof(float));
#else
SherpaOnnxCircularBufferPush(buf, data.Data(), data.ElementLength());
#endif
}
// see https://github.com/nodejs/node-addon-api/blob/main/doc/typed_array.md
... ... @@ -287,6 +294,17 @@ static SherpaOnnxSileroVadModelConfig GetSileroVadConfig(
static Napi::External<SherpaOnnxVoiceActivityDetector>
CreateVoiceActivityDetectorWrapper(const Napi::CallbackInfo &info) {
Napi::Env env = info.Env();
#if __OHOS__
// the last argument is a NativeResourceManager
if (info.Length() != 3) {
std::ostringstream os;
os << "Expect only 3 arguments. Given: " << info.Length();
Napi::TypeError::New(env, os.str()).ThrowAsJavaScriptException();
return {};
}
#else
if (info.Length() != 2) {
std::ostringstream os;
os << "Expect only 2 arguments. Given: " << info.Length();
... ... @@ -295,6 +313,7 @@ CreateVoiceActivityDetectorWrapper(const Napi::CallbackInfo &info) {
return {};
}
#endif
if (!info[0].IsObject()) {
Napi::TypeError::New(env,
... ... @@ -333,8 +352,15 @@ CreateVoiceActivityDetectorWrapper(const Napi::CallbackInfo &info) {
float buffer_size_in_seconds = info[1].As<Napi::Number>().FloatValue();
#if __OHOS__
std::unique_ptr<NativeResourceManager, decltype(&OH_ResourceManager_ReleaseNativeResourceManager)> mgr(OH_ResourceManager_InitNativeResourceManager(env, info[2]), &OH_ResourceManager_ReleaseNativeResourceManager);
SherpaOnnxVoiceActivityDetector *vad =
SherpaOnnxCreateVoiceActivityDetectorOHOS(&c, buffer_size_in_seconds, mgr.get());
#else
SherpaOnnxVoiceActivityDetector *vad =
SherpaOnnxCreateVoiceActivityDetector(&c, buffer_size_in_seconds);
#endif
if (c.silero_vad.model) {
delete[] c.silero_vad.model;
... ... @@ -383,8 +409,14 @@ static void VoiceActivityDetectorAcceptWaveformWrapper(
Napi::Float32Array samples = info[1].As<Napi::Float32Array>();
#if __OHOS__
// Note(fangjun): For unknown reasons, we need to use `/sizeof(float)` here for Huawei
SherpaOnnxVoiceActivityDetectorAcceptWaveform(vad, samples.Data(),
samples.ElementLength() / sizeof(float));
#else
SherpaOnnxVoiceActivityDetectorAcceptWaveform(vad, samples.Data(),
samples.ElementLength());
#endif
}
static Napi::Boolean VoiceActivityDetectorEmptyWrapper(
... ...
... ... @@ -85,7 +85,87 @@ static Napi::Object ReadWaveWrapper(const Napi::CallbackInfo &info) {
}
}
static Napi::Object ReadWaveFromBinaryWrapper(const Napi::CallbackInfo &info) {
Napi::Env env = info.Env();
if (info.Length() > 2) {
std::ostringstream os;
os << "Expect only 1 or 2 arguments. Given: " << info.Length();
Napi::TypeError::New(env, os.str()).ThrowAsJavaScriptException();
return {};
}
if (!info[0].IsTypedArray()) {
Napi::TypeError::New(env, "Argument 0 should be a float32 array")
.ThrowAsJavaScriptException();
return {};
}
Napi::Uint8Array data = info[0].As<Napi::Uint8Array>();
int32_t n = data.ElementLength();
const SherpaOnnxWave *wave = SherpaOnnxReadWaveFromBinaryData(reinterpret_cast<const char*>(data.Data()), n);
if (!wave) {
std::ostringstream os;
os << "Failed to read wave";
Napi::TypeError::New(env, os.str()).ThrowAsJavaScriptException();
return {};
}
bool enable_external_buffer = true;
if (info.Length() == 2) {
if (info[1].IsBoolean()) {
enable_external_buffer = info[1].As<Napi::Boolean>().Value();
} else {
Napi::TypeError::New(env, "Argument 1 should be a boolean")
.ThrowAsJavaScriptException();
return {};
}
}
if (enable_external_buffer) {
Napi::ArrayBuffer arrayBuffer = Napi::ArrayBuffer::New(
env, const_cast<float *>(wave->samples),
sizeof(float) * wave->num_samples,
[](Napi::Env /*env*/, void * /*data*/, const SherpaOnnxWave *hint) {
SherpaOnnxFreeWave(hint);
},
wave);
Napi::Float32Array float32Array =
Napi::Float32Array::New(env, wave->num_samples, arrayBuffer, 0);
Napi::Object obj = Napi::Object::New(env);
obj.Set(Napi::String::New(env, "samples"), float32Array);
obj.Set(Napi::String::New(env, "sampleRate"), wave->sample_rate);
return obj;
} else {
// don't use external buffer
Napi::ArrayBuffer arrayBuffer =
Napi::ArrayBuffer::New(env, sizeof(float) * wave->num_samples);
Napi::Float32Array float32Array =
Napi::Float32Array::New(env, wave->num_samples, arrayBuffer, 0);
std::copy(wave->samples, wave->samples + wave->num_samples,
float32Array.Data());
Napi::Object obj = Napi::Object::New(env);
obj.Set(Napi::String::New(env, "samples"), float32Array);
obj.Set(Napi::String::New(env, "sampleRate"), wave->sample_rate);
SherpaOnnxFreeWave(wave);
return obj;
}
}
void InitWaveReader(Napi::Env env, Napi::Object exports) {
exports.Set(Napi::String::New(env, "readWave"),
Napi::Function::New(env, ReadWaveWrapper));
}
exports.Set(Napi::String::New(env, "readWaveFromBinary"),
Napi::Function::New(env, ReadWaveFromBinaryWrapper));
}
\ No newline at end of file
... ...
import hilog from '@ohos.hilog';
import testNapi from 'libsherpa_onnx.so';
@Component
export struct MainPage {
@State message: string = 'Hello World';
build() {
Row() {
Column() {
Text(this.message)
.fontSize(50)
.fontWeight(FontWeight.Bold)
.onClick(() => {
})
}
.width('100%')
}
.height('100%')
}
}
\ No newline at end of file
... ...
import {
acceptWaveformOffline,
createOfflineRecognizer,
createOfflineStream,
decodeOfflineStream,
getOfflineStreamResultAsJson,
} from 'libsherpa_onnx.so';
export interface Samples {
samples: Float32Array;
sampleRate: number;
}
export class OfflineStream {
public handle: object;
constructor(handle: object) {
this.handle = handle;
}
// obj is {samples: samples, sampleRate: sampleRate}
// samples is a float32 array containing samples in the range [-1, 1]
// sampleRate is a number
acceptWaveform(obj: Samples) {
acceptWaveformOffline(this.handle, obj)
}
}
export class FeatureConfig {
public sampleRate: number = 16000;
public featureDim: number = 80;
}
export class OfflineTransducerModelConfig {
public encoder: string = '';
public decoder: string = '';
public joiner: string = '';
}
export class OfflineParaformerModelConfig {
public model: string = '';
}
export class OfflineNemoEncDecCtcModelConfig {
public model: string = '';
}
export class OfflineWhisperModelConfig {
public encoder: string = '';
public decoder: string = '';
public language: string = '';
public task: string = 'transcribe';
public tailPaddings: number = -1;
}
export class OfflineTdnnModelConfig {
public model: string = '';
}
export class OfflineSenseVoiceModelConfig {
public model: string = '';
public language: string = '';
public useItn: boolean = false;
}
export class OfflineMoonshineModelConfig {
public preprocessor: string = '';
public encoder: string = '';
public uncachedDecoder: string = '';
public cachedDecoder: string = '';
}
export class OfflineModelConfig {
public transducer: OfflineTransducerModelConfig = new OfflineTransducerModelConfig();
public paraformer: OfflineParaformerModelConfig = new OfflineParaformerModelConfig();
public nemoCtc: OfflineNemoEncDecCtcModelConfig = new OfflineNemoEncDecCtcModelConfig();
public whisper: OfflineWhisperModelConfig = new OfflineWhisperModelConfig();
public tdnn: OfflineTdnnModelConfig = new OfflineTdnnModelConfig();
public tokens: string = '';
public numThreads: number = 1;
public debug: boolean = false;
public provider: string = "cpu";
public modelType: string = '';
public modelingUnit: string = "cjkchar";
public bpeVocab: string = '';
public telespeechCtc: string = '';
public senseVoice: OfflineSenseVoiceModelConfig = new OfflineSenseVoiceModelConfig();
public moonshine: OfflineMoonshineModelConfig = new OfflineMoonshineModelConfig();
}
export class OfflineLMConfig {
public model: string = '';
public scale: number = 1.0;
}
export class OfflineRecognizerConfig {
public featConfig: FeatureConfig = new FeatureConfig();
public modelConfig: OfflineModelConfig = new OfflineModelConfig();
public lmConfig: OfflineLMConfig = new OfflineLMConfig();
public decodingMethod: string = "greedy_search";
public maxActivePaths: number = 4;
public hotwordsFfile: string = '';
public hotwordsScore: number = 1.5;
public ruleFsts: string = '';
public ruleFars: string = '';
public blankPenalty: number = 0;
}
export class OfflineRecognizerResult {
public text: string = '';
public timestamps: number[] = [];
public tokens: string[] = [];
public json = '';
public lang: string = '';
public emotion: string = '';
public event: string = '';
}
interface OfflineRecognizerResultJson {
text: string;
timestamps: number[];
tokens: string[];
lang: string;
emotion: string;
event: string;
}
export class OfflineRecognizer {
public handle: object;
public config: OfflineRecognizerConfig;
constructor(config: OfflineRecognizerConfig, mgr?: object) {
this.handle = createOfflineRecognizer(config, mgr);
this.config = config
}
createStream(): OfflineStream {
const handle: object = createOfflineStream(this.handle);
return new OfflineStream(handle);
}
decode(stream: OfflineStream) {
decodeOfflineStream(this.handle, stream.handle);
}
getResult(stream: OfflineStream): OfflineRecognizerResult {
const jsonStr: string = getOfflineStreamResultAsJson(stream.handle);
let o = JSON.parse(jsonStr) as OfflineRecognizerResultJson;
const r = new OfflineRecognizerResult()
r.text = o.text
r.timestamps = o.timestamps;
r.tokens = o.tokens;
r.json = jsonStr;
r.lang = o.lang;
r.emotion = o.emotion;
r.event = o.event;
return r;
}
}
\ No newline at end of file
... ...
import {
acceptWaveformOnline,
createOnlineRecognizer,
createOnlineStream,
decodeOnlineStream,
getOnlineStreamResultAsJson,
inputFinished,
isEndpoint,
isOnlineStreamReady,
reset,
} from 'libsherpa_onnx.so';
import { FeatureConfig, Samples } from './NonStreamingAsr';
export class OnlineStream {
public handle: object;
constructor(handle: object) {
this.handle = handle;
}
// obj is {samples: samples, sampleRate: sampleRate}
// samples is a float32 array containing samples in the range [-1, 1]
// sampleRate is a number
acceptWaveform(obj: Samples) {
acceptWaveformOnline(this.handle, obj)
}
inputFinished() {
inputFinished(this.handle)
}
}
export class OnlineTransducerModelConfig {
public encoder: string = '';
public decoder: string = '';
public joiner: string = '';
}
export class OnlineParaformerModelConfig {
public encoder: string = '';
public decoder: string = '';
}
export class OnlineZipformer2CtcModelConfig {
public model: string = '';
}
export class OnlineModelConfig {
public transducer: OnlineTransducerModelConfig = new OnlineTransducerModelConfig();
public paraformer: OnlineParaformerModelConfig = new OnlineParaformerModelConfig();
public zipformer2_ctc: OnlineZipformer2CtcModelConfig = new OnlineZipformer2CtcModelConfig();
public tokens: string = '';
public numThreads: number = 1;
public provider: string = "cpu";
public debug: boolean = false;
public modelType: string = '';
public modelingUnit: string = "cjkchar";
public bpeVocab: string = '';
}
export class OnlineCtcFstDecoderConfig {
public graph: string = '';
public maxActive: number = 3000;
}
export class OnlineRecognizerConfig {
public featConfig: FeatureConfig = new FeatureConfig();
public modelConfig: OnlineModelConfig = new OnlineModelConfig();
public decodingMethod: string = "greedy_search";
public maxActivePaths: number = 4;
public enableEndpoint: boolean = false;
public rule1MinTrailingSilence: number = 2.4;
public rule2MinTrailingSilence: number = 1.2;
public rule3MinUtteranceLength: number = 20;
public hotwordsFile: string = '';
public hotwordsScore: number = 1.5;
public ctcFstDecoderConfig: OnlineCtcFstDecoderConfig = new OnlineCtcFstDecoderConfig();
public ruleFsts: string = '';
public ruleFars: string = '';
public blankPenalty: number = 0;
}
interface OnlineRecognizerResultJson {
text: string;
timestamps: number[];
tokens: string[];
}
export class OnlineRecognizerResult {
public text: string = '';
public tokens: string[] = [];
public timestamps: number[] = [];
public json: string = '';
}
export class OnlineRecognizer {
public handle: object;
public config: OnlineRecognizerConfig
constructor(config: OnlineRecognizerConfig, mgr?: object) {
this.handle = createOnlineRecognizer(config, mgr);
this.config = config
}
createStream(): OnlineStream {
const handle: object = createOnlineStream(this.handle);
return new OnlineStream(handle);
}
isReady(stream: OnlineStream): boolean {
return isOnlineStreamReady(this.handle, stream.handle);
}
decode(stream: OnlineStream) {
decodeOnlineStream(this.handle, stream.handle);
}
isEndpoint(stream: OnlineStream): boolean {
return isEndpoint(this.handle, stream.handle);
}
reset(stream: OnlineStream) {
reset(this.handle, stream.handle);
}
getResult(stream: OnlineStream): OnlineRecognizerResult {
const jsonStr: string =
getOnlineStreamResultAsJson(this.handle, stream.handle);
let o = JSON.parse(jsonStr) as OnlineRecognizerResultJson;
const r = new OnlineRecognizerResult()
r.text = o.text
r.timestamps = o.timestamps;
r.tokens = o.tokens;
r.json = jsonStr;
return r;
}
}
\ No newline at end of file
... ...
import {
circularBufferGet,
circularBufferHead,
circularBufferPop,
circularBufferPush,
circularBufferReset,
circularBufferSize,
createCircularBuffer,
createVoiceActivityDetector,
voiceActivityDetectorAcceptWaveform,
voiceActivityDetectorClear,
voiceActivityDetectorFlush,
voiceActivityDetectorFront,
voiceActivityDetectorIsDetected,
voiceActivityDetectorIsEmpty,
voiceActivityDetectorPop,
voiceActivityDetectorReset,
} from 'libsherpa_onnx.so';
export class SileroVadConfig {
public model: string;
public threshold: number;
public minSpeechDuration: number;
public minSilenceDuration: number;
public windowSize: number;
public constructor(model: string, threshold: number, minSpeechDuration: number, minSilenceDuration: number,
windowSize: number) {
this.model = model;
this.threshold = threshold;
this.minSpeechDuration = minSpeechDuration;
this.minSilenceDuration = minSilenceDuration;
this.windowSize = windowSize;
}
}
export class VadConfig {
public sileroVad: SileroVadConfig;
public sampleRate: number;
public debug: boolean;
public numThreads: number;
public constructor(sileroVad: SileroVadConfig, sampleRate: number, debug: boolean, numThreads: number) {
this.sileroVad = sileroVad;
this.sampleRate = sampleRate;
this.debug = debug;
this.numThreads = numThreads;
}
}
export class CircularBuffer {
private handle: object;
constructor(capacity: number) {
this.handle = createCircularBuffer(capacity);
}
// samples is a float32 array
push(samples: Float32Array) {
console.log(`here samples: ${samples}`);
circularBufferPush(this.handle, samples);
}
// return a float32 array
get(startIndex: number, n: number, enableExternalBuffer: boolean = true): Float32Array {
return circularBufferGet(
this.handle, startIndex, n, enableExternalBuffer);
}
pop(n: number) {
circularBufferPop(this.handle, n);
}
size(): number {
return circularBufferSize(this.handle);
}
head(): number {
return circularBufferHead(this.handle);
}
reset() {
circularBufferReset(this.handle);
}
}
export interface SpeechSegment {
samples: Float32Array;
start: number;
}
export class Vad {
public config: VadConfig;
private handle: object;
constructor(config: VadConfig, bufferSizeInSeconds?: number, mgr?: object) {
this.handle =
createVoiceActivityDetector(config, bufferSizeInSeconds, mgr);
this.config = config;
}
acceptWaveform(samples: Float32Array): void {
voiceActivityDetectorAcceptWaveform(this.handle, samples);
}
isEmpty(): boolean {
return voiceActivityDetectorIsEmpty(this.handle);
}
isDetected(): boolean {
return voiceActivityDetectorIsDetected(this.handle);
}
pop(): void {
voiceActivityDetectorPop(this.handle);
}
clear(): void {
voiceActivityDetectorClear(this.handle);
}
front(enableExternalBuffer = true): SpeechSegment {
return voiceActivityDetectorFront(this.handle, enableExternalBuffer);
}
reset(): void {
voiceActivityDetectorReset(this.handle);
}
flush(): void {
voiceActivityDetectorFlush(this.handle);
}
}
\ No newline at end of file
... ...
{
"module": {
"name": "sherpa_onnx",
"type": "har",
"deviceTypes": [
"default",
"tablet",
"2in1"
]
}
}
... ...
{
"string": [
{
"name": "page_show",
"value": "page from package"
}
]
}
... ...
{
"string": [
{
"name": "page_show",
"value": "page from package"
}
]
}
... ...
{
"string": [
{
"name": "page_show",
"value": "page from package"
}
]
}
... ...
import hilog from '@ohos.hilog';
import { describe, beforeAll, beforeEach, afterEach, afterAll, it, expect } from '@ohos/hypium';
export default function abilityTest() {
describe('ActsAbilityTest', () => {
// Defines a test suite. Two parameters are supported: test suite name and test suite function.
beforeAll(() => {
// Presets an action, which is performed only once before all test cases of the test suite start.
// This API supports only one parameter: preset action function.
})
beforeEach(() => {
// Presets an action, which is performed before each unit test case starts.
// The number of execution times is the same as the number of test cases defined by **it**.
// This API supports only one parameter: preset action function.
})
afterEach(() => {
// Presets a clear action, which is performed after each unit test case ends.
// The number of execution times is the same as the number of test cases defined by **it**.
// This API supports only one parameter: clear action function.
})
afterAll(() => {
// Presets a clear action, which is performed after all test cases of the test suite end.
// This API supports only one parameter: clear action function.
})
it('assertContain', 0, () => {
// Defines a test case. This API supports three parameters: test case name, filter parameter, and test case function.
hilog.info(0x0000, 'testTag', '%{public}s', 'it begin');
let a = 'abc';
let b = 'b';
// Defines a variety of assertion methods, which are used to declare expected boolean conditions.
expect(a).assertContain(b);
expect(a).assertEqual(a);
})
})
}
\ No newline at end of file
... ...
import abilityTest from './Ability.test';
export default function testsuite() {
abilityTest();
}
\ No newline at end of file
... ...
{
"module": {
"name": "sherpa_onnx_test",
"type": "feature",
"deviceTypes": [
"default",
"tablet",
"2in1"
],
"deliveryWithInstall": true,
"installationFree": false
}
}
... ...
import localUnitTest from './LocalUnit.test';
export default function testsuite() {
localUnitTest();
}
\ No newline at end of file
... ...
import { describe, beforeAll, beforeEach, afterEach, afterAll, it, expect } from '@ohos/hypium';
export default function localUnitTest() {
describe('localUnitTest', () => {
// Defines a test suite. Two parameters are supported: test suite name and test suite function.
beforeAll(() => {
// Presets an action, which is performed only once before all test cases of the test suite start.
// This API supports only one parameter: preset action function.
});
beforeEach(() => {
// Presets an action, which is performed before each unit test case starts.
// The number of execution times is the same as the number of test cases defined by **it**.
// This API supports only one parameter: preset action function.
});
afterEach(() => {
// Presets a clear action, which is performed after each unit test case ends.
// The number of execution times is the same as the number of test cases defined by **it**.
// This API supports only one parameter: clear action function.
});
afterAll(() => {
// Presets a clear action, which is performed after all test cases of the test suite end.
// This API supports only one parameter: clear action function.
});
it('assertContain', 0, () => {
// Defines a test case. This API supports three parameters: test case name, filter parameter, and test case function.
let a = 'abc';
let b = 'b';
// Defines a variety of assertion methods, which are used to declare expected boolean conditions.
expect(a).assertContain(b);
expect(a).assertEqual(a);
});
});
}
\ No newline at end of file
... ...
../../../harmony-os/SherpaOnnxHar/sherpa_onnx/src/main/cpp/audio-tagging.cc
\ No newline at end of file
... ...
../../../harmony-os/SherpaOnnxHar/sherpa_onnx/src/main/cpp/keyword-spotting.cc
\ No newline at end of file
... ...
../../../harmony-os/SherpaOnnxHar/sherpa_onnx/src/main/cpp/macros.h
\ No newline at end of file
... ...
../../../harmony-os/SherpaOnnxHar/sherpa_onnx/src/main/cpp/non-streaming-asr.cc
\ No newline at end of file
... ...
../../../harmony-os/SherpaOnnxHar/sherpa_onnx/src/main/cpp/non-streaming-speaker-diarization.cc
\ No newline at end of file
... ...
../../../harmony-os/SherpaOnnxHar/sherpa_onnx/src/main/cpp/non-streaming-tts.cc
\ No newline at end of file
... ...
../../../harmony-os/SherpaOnnxHar/sherpa_onnx/src/main/cpp/punctuation.cc
\ No newline at end of file
... ...
../../../harmony-os/SherpaOnnxHar/sherpa_onnx/src/main/cpp/sherpa-onnx-node-addon-api.cc
\ No newline at end of file
... ...
../../../harmony-os/SherpaOnnxHar/sherpa_onnx/src/main/cpp/speaker-identification.cc
\ No newline at end of file
... ...
../../../harmony-os/SherpaOnnxHar/sherpa_onnx/src/main/cpp/spoken-language-identification.cc
\ No newline at end of file
... ...
../../../harmony-os/SherpaOnnxHar/sherpa_onnx/src/main/cpp/streaming-asr.cc
\ No newline at end of file
... ...
../../../harmony-os/SherpaOnnxHar/sherpa_onnx/src/main/cpp/vad.cc
\ No newline at end of file
... ...
../../../harmony-os/SherpaOnnxHar/sherpa_onnx/src/main/cpp/wave-reader.cc
\ No newline at end of file
... ...
../../../harmony-os/SherpaOnnxHar/sherpa_onnx/src/main/cpp/wave-writer.cc
\ No newline at end of file
... ...
... ... @@ -8,6 +8,7 @@
#include <cstring>
#include <memory>
#include <string>
#include <strstream>
#include <utility>
#include <vector>
... ... @@ -1201,6 +1202,28 @@ const SherpaOnnxWave *SherpaOnnxReadWave(const char *filename) {
return wave;
}
const SherpaOnnxWave *SherpaOnnxReadWaveFromBinaryData(const char *data,
int32_t n) {
int32_t sample_rate = -1;
bool is_ok = false;
std::istrstream is(data, n);
std::vector<float> samples = sherpa_onnx::ReadWave(is, &sample_rate, &is_ok);
if (!is_ok) {
return nullptr;
}
float *c_samples = new float[samples.size()];
std::copy(samples.begin(), samples.end(), c_samples);
SherpaOnnxWave *wave = new SherpaOnnxWave;
wave->samples = c_samples;
wave->sample_rate = sample_rate;
wave->num_samples = samples.size();
return wave;
}
void SherpaOnnxFreeWave(const SherpaOnnxWave *wave) {
if (wave) {
delete[] wave->samples;
... ...
... ... @@ -1001,6 +1001,14 @@ SHERPA_ONNX_API typedef struct SherpaOnnxWave {
// SherpaOnnxFreeWave() to free the returned pointer to avoid memory leak.
SHERPA_ONNX_API const SherpaOnnxWave *SherpaOnnxReadWave(const char *filename);
// Similar to SherpaOnnxReadWave(), it has read the content of `filename`
// into the array `data`.
//
// If the returned pointer is not NULL, the user has to invoke
// SherpaOnnxFreeWave() to free the returned pointer to avoid memory leak.
SHERPA_ONNX_API const SherpaOnnxWave *SherpaOnnxReadWaveFromBinaryData(
const char *data, int32_t n);
SHERPA_ONNX_API void SherpaOnnxFreeWave(const SherpaOnnxWave *wave);
// ============================================================
... ...