Android에서 Huawei ML Kit를 사용하여 카메라 스트림에서 온 디바이스 텍스트 감지 및 번역[Navigation Components, MVVM]
소개
이 기사에서는 Huawei ML 키트 카메라 스트림을 Android 애플리케이션 KnowMyBoard에 통합하는 방법을 알아봅니다. Account Kit는 대규모 사용자 기반의 앱에 원활한 로그인 기능을 제공합니다.
텍스트 인식 서비스는 영수증, 명함, 문서 이미지에서 텍스트를 추출할 수 있습니다. 이 서비스는 인쇄, 교육 및 물류와 같은 산업에 유용합니다. 이를 사용하여 데이터 입력을 처리하고 작업을 확인하는 앱을 만들 수 있습니다.
텍스트 인식 서비스는 동기식 또는 비동기식으로 호출하여 텍스트 인식 지원 앱을 구축할 수 있는 API 호스트를 사용하여 정적 이미지와 동적 카메라 스트림 모두에서 텍스트를 인식할 수 있습니다.
온디바이스 언어 감지 서비스는 인터넷을 사용할 수 없을 때 텍스트의 언어를 감지할 수 있습니다. ML Kit는 텍스트에서 언어를 감지하고 언어 코드(ISO 639-1 표준을 준수함)와 해당 신뢰도 또는 신뢰도가 가장 높은 언어 코드를 반환합니다. 현재 56개 언어를 감지할 수 있습니다.
실시간 번역 서비스와 마찬가지로 온디바이스 번역 서비스는 다른 언어 간의 번역이 필요한 시나리오에서 널리 사용될 수 있습니다. 예를 들어, 여행 앱은 이 서비스를 통합하여 다른 언어로 된 도로 표지판과 메뉴를 관광객의 모국어로 번역하여 더 세심한 서비스를 제공할 수 있습니다. 실시간 번역과 달리 온디바이스 번역은 인터넷 연결이 필요하지 않습니다. 인터넷이 끊긴 상태에서도 번역 서비스를 쉽게 이용하실 수 있습니다.
지침
개발 개요
Android Studio IDE를 설치해야 하며 Android 애플리케이션 개발에 대한 사전 지식이 있다고 가정합니다.
하드웨어 요구 사항
Windows 10을 실행하는 컴퓨터(데스크탑 또는 노트북).
디버깅에 사용되는 Android 휴대폰(USB 케이블 포함).
소프트웨어 요구 사항
자바 JDK 1.8 이상.
Android Studio 소프트웨어 또는 Visual Studio 또는 Code가 설치되었습니다.
HMS 코어(APK) 4.X 이상
통합 단계
1단계. Huawei 개발자 계정 및 Huawei 개발자 웹사이트에서 신원 확인을 완료하려면 Huawei ID 등록을 참조하십시오.
2단계.AppGallery Connect에서 프로젝트 생성
3단계. HMS Core SDK 추가
코딩을 시작하자
navigation_graph.xml
<?xml version="1.0" encoding="utf-8"?>
<navigation xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:id="@+id/navigation_graph"
app:startDestination="@id/loginFragment">
<fragment
android:id="@+id/loginFragment"
android:name="com.huawei.hms.knowmyboard.dtse.activity.fragments.LoginFragment"
android:label="LoginFragment"/>
<fragment
android:id="@+id/mainFragment"
android:name="com.huawei.hms.knowmyboard.dtse.activity.fragments.MainFragment"
android:label="MainFragment"/>
<fragment
android:id="@+id/searchFragment"
android:name="com.huawei.hms.knowmyboard.dtse.activity.fragments.SearchFragment"
android:label="fragment_search"
tools:layout="@layout/fragment_search" />
</navigation>
TextRecognitionActivity.java
public final class TextRecognitionActivity extends BaseActivity
implements OnRequestPermissionsResultCallback, View.OnClickListener {
private static final String TAG = "TextRecognitionActivity";
private LensEngine lensEngine = null;
private LensEnginePreview preview;
private GraphicOverlay graphicOverlay;
private ImageButton takePicture;
private ImageButton imageSwitch;
private RelativeLayout zoomImageLayout;
private ZoomImageView zoomImageView;
private ImageButton zoomImageClose;
CameraConfiguration cameraConfiguration = null;
private int facing = CameraConfiguration.CAMERA_FACING_BACK;
private Camera mCamera;
private boolean isLandScape;
private Bitmap bitmap;
private Bitmap bitmapCopy;
private LocalTextTransactor localTextTransactor;
private Handler mHandler = new MsgHandler(this);
private Dialog languageDialog;
private AddPictureDialog addPictureDialog;
private TextView textCN;
private TextView textEN;
private TextView textJN;
private TextView textKN;
private TextView textLN;
private TextView tv_language,tv_translated_txt;
private String textType = Constant.POSITION_CN;
private boolean isInitialization = false;
MLTextAnalyzer analyzer;
private static class MsgHandler extends Handler {
WeakReference<TextRecognitionActivity> mMainActivityWeakReference;
public MsgHandler(TextRecognitionActivity mainActivity) {
this.mMainActivityWeakReference = new WeakReference<>(mainActivity);
}
@Override
public void handleMessage(Message msg) {
super.handleMessage(msg);
TextRecognitionActivity mainActivity = this.mMainActivityWeakReference.get();
if (mainActivity == null) {
return;
}
if (msg.what == Constant.SHOW_TAKE_PHOTO_BUTTON) {
mainActivity.setVisible();
} else if (msg.what == Constant.HIDE_TAKE_PHOTO_BUTTON) {
mainActivity.setGone();
}
}
}
private void setVisible() {
if (this.takePicture.getVisibility() == View.GONE) {
this.takePicture.setVisibility(View.VISIBLE);
}
}
private void setGone() {
if (this.takePicture.getVisibility() == View.VISIBLE) {
this.takePicture.setVisibility(View.GONE);
}
}
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
this.setContentView(R.layout.activity_text_recognition);
if (savedInstanceState != null) {
this.facing = savedInstanceState.getInt(Constant.CAMERA_FACING);
}
this.tv_language = this.findViewById(R.id.tv_lang);
this.tv_translated_txt = this.findViewById(R.id.tv_translated_txt);
this.preview = this.findViewById(R.id.live_preview);
this.graphicOverlay = this.findViewById(R.id.live_overlay);
this.cameraConfiguration = new CameraConfiguration();
this.cameraConfiguration.setCameraFacing(this.facing);
this.initViews();
this.isLandScape = (this.getResources().getConfiguration().orientation == Configuration.ORIENTATION_LANDSCAPE);
this.createLensEngine();
this.setStatusBar();
}
private void initViews() {
this.takePicture = this.findViewById(R.id.takePicture);
this.takePicture.setOnClickListener(this);
this.imageSwitch = this.findViewById(R.id.text_imageSwitch);
this.imageSwitch.setOnClickListener(this);
this.zoomImageLayout = this.findViewById(R.id.zoomImageLayout);
this.zoomImageView = this.findViewById(R.id.take_picture_overlay);
this.zoomImageClose = this.findViewById(R.id.zoomImageClose);
this.zoomImageClose.setOnClickListener(this);
this.findViewById(R.id.back).setOnClickListener(this);
this.findViewById(R.id.language_setting).setOnClickListener(this);
this.createLanguageDialog();
this.createAddPictureDialog();
}
@Override
public void onClick(View view) {
if (view.getId() == R.id.takePicture) {
this.takePicture();
} else if (view.getId() == R.id.zoomImageClose) {
this.zoomImageLayout.setVisibility(View.GONE);
this.recycleBitmap();
} else if (view.getId() == R.id.text_imageSwitch) {
this.showAddPictureDialog();
} else if (view.getId() == R.id.language_setting) {
this.showLanguageDialog();
} else if (view.getId() == R.id.simple_cn) {
SharedPreferencesUtil.getInstance(this)
.putStringValue(Constant.POSITION_KEY, Constant.POSITION_CN);
this.languageDialog.dismiss();
this.restartLensEngine(Constant.POSITION_CN);
} else if (view.getId() == R.id.english) {
SharedPreferencesUtil.getInstance(this)
.putStringValue(Constant.POSITION_KEY, Constant.POSITION_EN);
this.languageDialog.dismiss();
this.preview.release();
this.restartLensEngine(Constant.POSITION_EN);
} else if (view.getId() == R.id.japanese) {
SharedPreferencesUtil.getInstance(this)
.putStringValue(Constant.POSITION_KEY, Constant.POSITION_JA);
this.languageDialog.dismiss();
this.preview.release();
this.restartLensEngine(Constant.POSITION_JA);
} else if (view.getId() == R.id.korean) {
SharedPreferencesUtil.getInstance(this)
.putStringValue(Constant.POSITION_KEY, Constant.POSITION_KO);
this.languageDialog.dismiss();
this.preview.release();
this.restartLensEngine(Constant.POSITION_KO);
} else if (view.getId() == R.id.latin) {
SharedPreferencesUtil.getInstance(this)
.putStringValue(Constant.POSITION_KEY, Constant.POSITION_LA);
this.languageDialog.dismiss();
this.preview.release();
this.restartLensEngine(Constant.POSITION_LA);
} else if (view.getId() == R.id.back) {
releaseLensEngine();
this.finish();
}
}
private void restartLensEngine(String type) {
if (this.textType.equals(type)) {
return;
}
this.lensEngine.release();
this.lensEngine = null;
this.createLensEngine();
this.startLensEngine();
if (this.lensEngine == null || this.lensEngine.getCamera() == null) {
return;
}
this.mCamera = this.lensEngine.getCamera();
try {
this.mCamera.setPreviewDisplay(this.preview.getSurfaceHolder());
} catch (IOException e) {
Log.d(TextRecognitionActivity.TAG, "initViews IOException");
}
}
@Override
public void onBackPressed() {
if (this.zoomImageLayout.getVisibility() == View.VISIBLE) {
this.zoomImageLayout.setVisibility(View.GONE);
this.recycleBitmap();
} else {
super.onBackPressed();
releaseLensEngine();
}
}
private void createLanguageDialog() {
this.languageDialog = new Dialog(this, R.style.MyDialogStyle);
View view = View.inflate(this, R.layout.dialog_language_setting, null);
// Set up a custom layout
this.languageDialog.setContentView(view);
this.textCN = view.findViewById(R.id.simple_cn);
this.textCN.setOnClickListener(this);
this.textEN = view.findViewById(R.id.english);
this.textEN.setOnClickListener(this);
this.textJN = view.findViewById(R.id.japanese);
this.textJN.setOnClickListener(this);
this.textKN = view.findViewById(R.id.korean);
this.textKN.setOnClickListener(this);
this.textLN = view.findViewById(R.id.latin);
this.textLN.setOnClickListener(this);
this.languageDialog.setCanceledOnTouchOutside(true);
// Set the size of the dialog
Window dialogWindow = this.languageDialog.getWindow();
WindowManager.LayoutParams layoutParams = dialogWindow.getAttributes();
layoutParams.width = WindowManager.LayoutParams.MATCH_PARENT;
layoutParams.height = WindowManager.LayoutParams.WRAP_CONTENT;
layoutParams.gravity = Gravity.BOTTOM;
dialogWindow.setAttributes(layoutParams);
}
private void showLanguageDialog() {
this.initDialogViews();
this.languageDialog.show();
}
private void createAddPictureDialog() {
this.addPictureDialog = new AddPictureDialog(this, AddPictureDialog.TYPE_NORMAL);
final Intent intent = new Intent(TextRecognitionActivity.this, RemoteDetectionActivity.class);
intent.putExtra(Constant.MODEL_TYPE, Constant.CLOUD_TEXT_DETECTION);
this.addPictureDialog.setClickListener(new AddPictureDialog.ClickListener() {
@Override
public void takePicture() {
lensEngine.release();
isInitialization = false;
intent.putExtra(Constant.ADD_PICTURE_TYPE, Constant.TYPE_TAKE_PHOTO);
TextRecognitionActivity.this.startActivity(intent);
}
@Override
public void selectImage() {
intent.putExtra(Constant.ADD_PICTURE_TYPE, Constant.TYPE_SELECT_IMAGE);
TextRecognitionActivity.this.startActivity(intent);
}
@Override
public void doExtend() {
}
});
}
private void showAddPictureDialog() {
this.addPictureDialog.show();
}
private void initDialogViews() {
String position = SharedPreferencesUtil.getInstance(this).getStringValue(Constant.POSITION_KEY);
this.textType = position;
this.textCN.setSelected(false);
this.textEN.setSelected(false);
this.textJN.setSelected(false);
this.textLN.setSelected(false);
this.textKN.setSelected(false);
switch (position) {
case Constant.POSITION_CN:
this.textCN.setSelected(true);
break;
case Constant.POSITION_EN:
this.textEN.setSelected(true);
break;
case Constant.POSITION_LA:
this.textLN.setSelected(true);
break;
case Constant.POSITION_JA:
this.textJN.setSelected(true);
break;
case Constant.POSITION_KO:
this.textKN.setSelected(true);
break;
default:
}
}
@Override
protected void onSaveInstanceState(Bundle outState) {
outState.putInt(Constant.CAMERA_FACING, this.facing);
super.onSaveInstanceState(outState);
}
private void createLensEngine() {
MLLocalTextSetting setting = new MLLocalTextSetting.Factory()
.setOCRMode(MLLocalTextSetting.OCR_DETECT_MODE)
// Specify languages that can be recognized.
.setLanguage("ko")
.create();
analyzer = MLAnalyzerFactory.getInstance().getLocalTextAnalyzer(setting);
//analyzer = new MLTextAnalyzer.Factory(this).create();
if (this.lensEngine == null) {
this.lensEngine = new LensEngine(this, this.cameraConfiguration, this.graphicOverlay);
}
try {
this.localTextTransactor = new LocalTextTransactor(this.mHandler, this);
this.lensEngine.setMachineLearningFrameTransactor(this.localTextTransactor);
// this.lensEngine.setMachineLearningFrameTransactor((ImageTransactor) new ObjectAnalyzerTransactor());
isInitialization = true;
} catch (Exception e) {
Toast.makeText(
this,
"Can not create image transactor: " + e.getMessage(),
Toast.LENGTH_LONG)
.show();
}
}
private void startLensEngine() {
if (this.lensEngine != null) {
try {
this.preview.start(this.lensEngine, false);
} catch (IOException e) {
Log.e(TextRecognitionActivity.TAG, "Unable to start lensEngine.", e);
this.lensEngine.release();
this.lensEngine = null;
}
}
}
@Override
public void onResume() {
super.onResume();
if (!isInitialization){
createLensEngine();
}
this.startLensEngine();
}
@Override
protected void onStop() {
super.onStop();
this.preview.stop();
}
private void releaseLensEngine() {
if (this.lensEngine != null) {
this.lensEngine.release();
this.lensEngine = null;
}
recycleBitmap();
}
@Override
protected void onDestroy() {
super.onDestroy();
releaseLensEngine();
if (analyzer != null) {
try {
analyzer.stop();
} catch (IOException e) {
// Exception handling.
Log.e(TAG,"Error while releasing analyzer");
}
}
}
private void recycleBitmap() {
if (this.bitmap != null && !this.bitmap.isRecycled()) {
this.bitmap.recycle();
this.bitmap = null;
}
if (this.bitmapCopy != null && !this.bitmapCopy.isRecycled()) {
this.bitmapCopy.recycle();
this.bitmapCopy = null;
}
}
private void takePicture() {
this.zoomImageLayout.setVisibility(View.VISIBLE);
LocalDataProcessor localDataProcessor = new LocalDataProcessor();
localDataProcessor.setLandScape(this.isLandScape);
this.bitmap = BitmapUtils.getBitmap(this.localTextTransactor.getTransactingImage(), this.localTextTransactor.getTransactingMetaData());
float previewWidth = localDataProcessor.getMaxWidthOfImage(this.localTextTransactor.getTransactingMetaData());
float previewHeight = localDataProcessor.getMaxHeightOfImage(this.localTextTransactor.getTransactingMetaData());
if (this.isLandScape) {
previewWidth = localDataProcessor.getMaxHeightOfImage(this.localTextTransactor.getTransactingMetaData());
previewHeight = localDataProcessor.getMaxWidthOfImage(this.localTextTransactor.getTransactingMetaData());
}
this.bitmapCopy = Bitmap.createBitmap(this.bitmap).copy(Bitmap.Config.ARGB_8888, true);
Canvas canvas = new Canvas(this.bitmapCopy);
float min = Math.min(previewWidth, previewHeight);
float max = Math.max(previewWidth, previewHeight);
if (this.getResources().getConfiguration().orientation == Configuration.ORIENTATION_PORTRAIT) {
localDataProcessor.setCameraInfo(this.graphicOverlay, canvas, min, max);
} else {
localDataProcessor.setCameraInfo(this.graphicOverlay, canvas, max, min);
}
localDataProcessor.drawHmsMLVisionText(canvas, this.localTextTransactor.getLastResults().getBlocks());
this.zoomImageView.setImageBitmap(this.bitmapCopy);
// Create an MLFrame object using the bitmap, which is the image data in bitmap format.
MLFrame frame = MLFrame.fromBitmap(bitmap);
Task<MLText> task = analyzer.asyncAnalyseFrame(frame);
task.addOnSuccessListener(new OnSuccessListener<MLText>() {
@Override
public void onSuccess(MLText text) {
String detectText = text.getStringValue();
// Processing for successful recognition.
// Create a local language detector.
MLLangDetectorFactory factory = MLLangDetectorFactory.getInstance();
MLLocalLangDetectorSetting setting = new MLLocalLangDetectorSetting.Factory()
// Set the minimum confidence threshold for language detection.
.setTrustedThreshold(0.01f)
.create();
MLLocalLangDetector myLocalLangDetector = factory.getLocalLangDetector(setting);
Task<String> firstBestDetectTask = myLocalLangDetector.firstBestDetect(detectText);
firstBestDetectTask.addOnSuccessListener(new OnSuccessListener<String>() {
@Override
public void onSuccess(String languageDetected) {
// Processing logic for detection success.
Log.d("TAG", "Lang detect :" + languageDetected);
Log.d("TAG", " detectText :" + detectText);
translate(languageDetected,detectText);
}
}).addOnFailureListener(new OnFailureListener() {
@Override
public void onFailure(Exception e) {
// Processing logic for detection failure.
Log.e("TAG", "Lang detect error:" + e.getMessage());
}
});
}
}).addOnFailureListener(new OnFailureListener() {
@Override
public void onFailure(Exception e) {
// Processing logic for recognition failure.
Log.e("TAG"," Text : Processing logic for recognition failure");
}
});
}
private void translate(String languageDetected, String detectText) {
MLApplication.initialize(getApplication());
MLApplication.getInstance().setApiKey("DAEDAF48ZIMI4ettQdTfCKlXgaln/E+TO/PrsX+LpP2BubkmED/iC0iVEps5vfx1ol27rHvuwiq64YphpPkGYWbf9La8XjnvC9qhwQ==");
// Create an offline translator.
MLLocalTranslateSetting setting = new MLLocalTranslateSetting.Factory()
// Set the source language code. The ISO 639-1 standard is used. This parameter is mandatory. If this parameter is not set, an error may occur.
.setSourceLangCode(languageDetected)
// Set the target language code. The ISO 639-1 standard is used. This parameter is mandatory. If this parameter is not set, an error may occur.
.setTargetLangCode("en")
.create();
MLLocalTranslator myLocalTranslator = MLTranslatorFactory.getInstance().getLocalTranslator(setting);
// Set the model download policy.
MLModelDownloadStrategy downloadStrategy = new MLModelDownloadStrategy.Factory()
.needWifi()// It is recommended that you download the package in a Wi-Fi environment.
.create();
// Create a download progress listener.
MLModelDownloadListener modelDownloadListener = new MLModelDownloadListener() {
@Override
public void onProcess(long alreadyDownLength, long totalLength) {
runOnUiThread(new Runnable() {
@Override
public void run() {
// Display the download progress or perform other operations.
}
});
}
};
myLocalTranslator.preparedModel(downloadStrategy, modelDownloadListener).
addOnSuccessListener(new OnSuccessListener<Void>() {
@Override
public void onSuccess(Void aVoid) {
// Called when the model package is successfully downloaded.
// input is a string of less than 5000 characters.
final Task<String> task = myLocalTranslator.asyncTranslate(detectText);
// Before translation, ensure that the models have been successfully downloaded.
task.addOnSuccessListener(new OnSuccessListener<String>() {
@Override
public void onSuccess(String translated) {
// Processing logic for detection success.
Log.e("TAG"," Translated Text : "+translated);
tv_translated_txt.setText(translated);
}
}).addOnFailureListener(new OnFailureListener() {
@Override
public void onFailure(Exception e) {
// Processing logic for detection failure.
Log.e("TAG"," Translation failed "+e.getMessage());
Toast.makeText(TextRecognitionActivity.this,"Please check internet connection.",Toast.LENGTH_SHORT).show();
}
});
}
}).addOnFailureListener(new OnFailureListener() {
@Override
public void onFailure(Exception e) {
// Called when the model package fails to be downloaded.
Log.e("TAG"," Translation failed onFailure "+e.getMessage());
}
});
}
}
MainFragment.java
public class MainFragment extends Fragment {
static String TAG = "TAG";
FragmentMainFragmentBinding binding;
LoginViewModel loginViewModel;
public MainFragment() {
// Required empty public constructor
}
@Override
public void onCreate(@Nullable Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setHasOptionsMenu(true);
}
@Override
public View onCreateView(LayoutInflater inflater, ViewGroup container,
Bundle savedInstanceState) {
// Inflate the layout for this fragment
binding = DataBindingUtil.inflate(inflater, R.layout.fragment_main_fragment, container, false);
loginViewModel = new ViewModelProvider(getActivity()).get(LoginViewModel.class);
binding.setLoginViewModel(loginViewModel);
binding.buttonScan.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View v) {
dialog();
}
});
loginViewModel.getImagePath().observeForever(new Observer<Bitmap>() {
@Override
public void onChanged(Bitmap bitmap) {
try{
binding.imageView.setImageBitmap(bitmap);
}catch (Exception e){
e.printStackTrace();
Log.e("TAG","Error : "+e.getMessage());
}
}
});
loginViewModel.getTextRecognized().observeForever(new Observer<ArrayList<String>>() {
@Override
public void onChanged(ArrayList<String> res) {
binding.textLanguage.setText("Language : "+getStringResourceByName(res.get(0)));
binding.textDetected.setText("Detected text : "+res.get(1));
binding.textTranslated.setText("Translated text : "+res.get(2));
}
});
return binding.getRoot();
}
private void openCamera() {
Intent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
getActivity().startActivityForResult(intent, OPEN_CAMERA);
}
private String getStringResourceByName(String aString) {
try{
String packageName = getActivity().getPackageName();
int resId = getResources()
.getIdentifier(aString, "string", packageName);
if (resId == 0) {
return aString;
} else {
return getString(resId);
}
}catch (Exception e){
e.printStackTrace();
return aString;
}
}
private void scan() {
Intent intent = new Intent(Intent.ACTION_GET_CONTENT);
intent.setType("image/*");
String[] mimeTypes = {"image/jpeg", "image/png"};
intent.putExtra(Intent.EXTRA_MIME_TYPES,mimeTypes);
getActivity().startActivityForResult(intent, OPEN_GALLERY);
}
public void dialog()
{
final Dialog dialog = new Dialog(getActivity(), R.style.AppTheme);
dialog.setTitle("Choose");
dialog.setContentView(R.layout.dialog_pop_up);
TextView txt_gallry=(TextView)dialog.findViewById(R.id.textView_gallery);
TextView txt_camera=(TextView)dialog.findViewById(R.id.textView_camera);
txt_gallry.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View v) {
dialog.dismiss();
scan();
}
});
txt_camera.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View v) {
dialog.dismiss();
openCamera();
}
});
dialog.show();
}
@Override
public void onCreateOptionsMenu(@NonNull Menu menu, @NonNull MenuInflater inflater) {
menu.clear();
super.onCreateOptionsMenu(menu, inflater);
inflater.inflate(R.menu.main_fragment_menu, menu);
}
@SuppressLint("NonConstantResourceId")
@Override
public boolean onOptionsItemSelected(MenuItem item) {
switch (item.getItemId()) {
case R.id.menu_camera:
getActivity().startActivityForResult(new Intent(getActivity(), TextRecognitionActivity.class),1234);
break;
}
return super.onOptionsItemSelected(item);
}
}
MainActivity.java
public class MainActivity extends AppCompatActivity {
LoginViewModel loginViewModel;
private MLTextAnalyzer mTextAnalyzer;
public Uri imagePath;
Bitmap bitmap;
static String TAG = "TAG";
ArrayList<String> result = new ArrayList<>();
MLLocalLangDetector myLocalLangDetector;
MLLocalTranslator myLocalTranslator;
String textRecognized;
ProgressDialog progressDialog;
NavController navController;
ActivityMainBinding activityMainBinding;
BottomNavigationView bottomNavigationView;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
activityMainBinding = DataBindingUtil.setContentView(MainActivity.this,R.layout.activity_main);
loginViewModel = new ViewModelProvider(MainActivity.this).get(LoginViewModel.class);
navController = Navigation.findNavController(MainActivity.this, R.id.nav_host_fragment);
MyApplication.setActivity(this);
progressDialog = new ProgressDialog(this);
progressDialog.setCancelable(false);
bottomNavigationView = activityMainBinding.bottomNavigation;
NavigationUI.setupWithNavController(bottomNavigationView, navController);
}
@Override
protected void onActivityResult(int requestCode, int resultCode, @Nullable Intent data) {
// Process the authorization result to obtain the authorization code from AuthAccount.
super.onActivityResult(requestCode, resultCode, data);
if (requestCode == 8888) {
Task<AuthAccount> authAccountTask = AccountAuthManager.parseAuthResultFromIntent(data);
if (authAccountTask.isSuccessful()) {
// The sign-in is successful, and the user's ID information and authorization code are obtained.
AuthAccount authAccount = authAccountTask.getResult();
UserData userData = new UserData();
userData.setAccessToken(authAccount.getAccessToken());
userData.setCountryCode(authAccount.getCountryCode());
userData.setDisplayName(authAccount.getDisplayName());
userData.setEmail(authAccount.getEmail());
userData.setFamilyName(authAccount.getFamilyName());
userData.setGivenName(authAccount.getGivenName());
userData.setIdToken(authAccount.getIdToken());
userData.setOpenId(authAccount.getOpenId());
userData.setUid(authAccount.getUid());
userData.setPhotoUriString(authAccount.getAvatarUri().toString());
userData.setUnionId(authAccount.getUnionId());
loginViewModel = new ViewModelProvider(MainActivity.this).get(LoginViewModel.class);
loginViewModel.sendData(authAccount.getDisplayName());
} else {
// The sign-in failed.
Log.e("TAG", "sign in failed:" + ((ApiException) authAccountTask.getException()).getStatusCode());
}
}
if (requestCode == 2323 && resultCode == RESULT_OK && data != null) {
progressDialog.setMessage("Initializing text detection..");
progressDialog.show();
imagePath = data.getData();
try {
bitmap = MediaStore.Images.Media.getBitmap(this.getContentResolver(), imagePath);
asyncAnalyzeText(bitmap);
} catch (IOException e) {
e.printStackTrace();
Log.e("TAG", " BITMAP ERROR");
}
}
if (requestCode == 2424 && resultCode == RESULT_OK && data != null) {
progressDialog.setMessage("Initializing text detection..");
progressDialog.show();
try {
bitmap = (Bitmap) data.getExtras().get("data");
asyncAnalyzeText(bitmap);
} catch (Exception e) {
e.printStackTrace();
Log.e("TAG", " BITMAP ERROR");
}
}
}
private void asyncAnalyzeText(Bitmap bitmap) {
if (mTextAnalyzer == null) {
createMLTextAnalyzer();
}
MLFrame frame = MLFrame.fromBitmap(bitmap);
Task<MLText> task = mTextAnalyzer.asyncAnalyseFrame(frame);
task.addOnSuccessListener(new OnSuccessListener<MLText>() {
@Override
public void onSuccess(MLText text) {
progressDialog.setMessage("Initializing language detection..");
textRecognized = text.getStringValue().trim();
if(!textRecognized.isEmpty()){
// Create a local language detector.
MLLangDetectorFactory factory = MLLangDetectorFactory.getInstance();
MLLocalLangDetectorSetting setting = new MLLocalLangDetectorSetting.Factory()
// Set the minimum confidence threshold for language detection.
.setTrustedThreshold(0.01f)
.create();
myLocalLangDetector = factory.getLocalLangDetector(setting);
Task<String> firstBestDetectTask = myLocalLangDetector.firstBestDetect(textRecognized);
firstBestDetectTask.addOnSuccessListener(new OnSuccessListener<String>() {
@Override
public void onSuccess(String languageDetected) {
progressDialog.setMessage("Initializing text translation..");
// Processing logic for detection success.
textTranslate(languageDetected, textRecognized, bitmap);
}
}).addOnFailureListener(new OnFailureListener() {
@Override
public void onFailure(Exception e) {
// Processing logic for detection failure.
Log.e("TAG", "Lang detect error:" + e.getMessage());
}
});
}else{
progressDialog.dismiss();
showErrorDialog("Failed to recognize text.");
}
}
}).addOnFailureListener(new OnFailureListener() {
@Override
public void onFailure(Exception e) {
Log.e("TAG", "#==>" + e.getMessage());
}
});
}
private void showErrorDialog(String msg) {
AlertDialog alertDialog = new AlertDialog.Builder(this).create();
alertDialog.setTitle("Error");
alertDialog.setMessage(msg);
alertDialog.setButton(AlertDialog.BUTTON_POSITIVE, "OK", new DialogInterface.OnClickListener() {
public void onClick(DialogInterface dialog, int which) {
dialog.dismiss();
}
});
alertDialog.show();
}
private void textTranslate(String languageDetected, String textRecognized, Bitmap uri) {
MLApplication.initialize(getApplication());
MLApplication.getInstance().setApiKey(Constants.API_KEY);
Log.d(TAG,"Lang detect : "+languageDetected);
Log.d(TAG,"Text : "+textRecognized);
// Create an offline translator.
MLLocalTranslateSetting setting = new MLLocalTranslateSetting.Factory()
// Set the source language code. The ISO 639-1 standard is used. This parameter is mandatory. If this parameter is not set, an error may occur.
.setSourceLangCode(languageDetected)
// Set the target language code. The ISO 639-1 standard is used. This parameter is mandatory. If this parameter is not set, an error may occur.
.setTargetLangCode("en")
.create();
myLocalTranslator = MLTranslatorFactory.getInstance().getLocalTranslator(setting);
// Set the model download policy.
MLModelDownloadStrategy downloadStrategy = new MLModelDownloadStrategy.Factory()
.needWifi()// It is recommended that you download the package in a Wi-Fi environment.
.create();
// Create a download progress listener.
MLModelDownloadListener modelDownloadListener = new MLModelDownloadListener() {
@Override
public void onProcess(long alreadyDownLength, long totalLength) {
runOnUiThread(new Runnable() {
@Override
public void run() {
// Display the download progress or perform other operations.
}
});
}
};
myLocalTranslator.preparedModel(downloadStrategy, modelDownloadListener).
addOnSuccessListener(new OnSuccessListener<Void>() {
@Override
public void onSuccess(Void aVoid) {
// Called when the model package is successfully downloaded.
// input is a string of less than 5000 characters.
final Task<String> task = myLocalTranslator.asyncTranslate(textRecognized);
// Before translation, ensure that the models have been successfully downloaded.
task.addOnSuccessListener(new OnSuccessListener<String>() {
@Override
public void onSuccess(String translated) {
// Processing logic for detection success.
result.clear();
result.add(languageDetected.trim());
result.add(textRecognized.trim());
result.add(translated.trim());
loginViewModel.setImage(uri);
loginViewModel.setTextRecognized(result);
progressDialog.dismiss();
}
}).addOnFailureListener(new OnFailureListener() {
@Override
public void onFailure(Exception e) {
// Processing logic for detection failure.
progressDialog.dismiss();
}
});
}
}).addOnFailureListener(new OnFailureListener() {
@Override
public void onFailure(Exception e) {
// Called when the model package fails to be downloaded.
progressDialog.dismiss();
}
});
}
private void createMLTextAnalyzer() {
MLLocalTextSetting setting = new MLLocalTextSetting.Factory()
.setOCRMode(MLLocalTextSetting.OCR_DETECT_MODE)
.create();
mTextAnalyzer = MLAnalyzerFactory.getInstance().getLocalTextAnalyzer(setting);
}
@Override
protected void onStop() {
if (myLocalLangDetector != null) {
myLocalLangDetector.stop();
}
if (myLocalTranslator!= null) {
myLocalTranslator.stop();
}
if(progressDialog!= null){
progressDialog.dismiss();
}
super.onStop();
}
}
결과
샘플 응용 프로그램 결과는 아래 gif에서 텍스트 텍스트 인식, 언어 감지 및 텍스트 번역을 보여줍니다. 일본어 텍스트 인식 및 감지 및 영어로의 번역을 볼 수 있습니다. 마찬가지로 애플리케이션에서 ML 서비스를 사용할 수 있습니다.
트릭과 팁
agconnect-services.json 파일이 추가되었는지 확인합니다.
필수 종속 항목이 추가되었는지 확인
AGC에서 서비스가 활성화되어 있는지 확인하십시오.
gradle.build 파일에서 데이터 바인딩 활성화
하단 탐색 ID가 탐색 그래프의 프래그먼트 ID와 동일해야 합니다.
서비스를 호출하기 전에 apk 키를 설정했는지 확인하십시오.
아래 링크에서 모듈 텍스트를 추가했는지 확인하십시오.
gradle 파일 애플리케이션에서 모듈 텍스트의 라이브러리로 변경
결론
이 기사에서는 장치 카메라 스트림에서 텍스트를 추출하고 언어를 감지하고 Android 애플리케이션 KnowMyBoard에서 영어로 번역할 수 있는 Huawei ML 키트 카메라 스트림을 통합하는 방법을 배웠습니다. 결과 섹션에서 원하는 결과를 확인할 수 있습니다. 여기에서 이전 기사를 살펴볼 수도 있습니다. Huawei Ml 키트 기능이 이 샘플과 같이 귀하에게도 도움이 되기를 바라며 귀하의 요구 사항에 따라 사용할 수 있습니다.
읽어주셔서 정말 감사합니다. 이 기사가 Android 애플리케이션 KnowMyBoard에 Huawei ML 키트 통합을 이해하는 데 도움이 되기를 바랍니다.
참조
Huawei ML Kit – Training video
ML Text Recognition
Module-text
Reference
이 문제에 관하여(Android에서 Huawei ML Kit를 사용하여 카메라 스트림에서 온 디바이스 텍스트 감지 및 번역[Navigation Components, MVVM]), 우리는 이곳에서 더 많은 자료를 발견하고 링크를 클릭하여 보았다 https://dev.to/hmscommunity/on-device-text-detection-and-translation-from-camera-stream-using-huawei-ml-kit-in-android-navigation-components-mvvm-2j0l텍스트를 자유롭게 공유하거나 복사할 수 있습니다.하지만 이 문서의 URL은 참조 URL로 남겨 두십시오.
우수한 개발자 콘텐츠 발견에 전념
(Collection and Share based on the CC Protocol.)
좋은 웹페이지 즐겨찾기
